-- Logs begin at Sat 2022-05-28 20:01:13 UTC, end at Sat 2022-05-28 22:29:24 UTC. -- May 28 20:52:55.561342 ncdv-master-1 systemd[1]: Started Devstack devstack@c-vol.service. May 28 20:52:58.925016 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=135745) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}} May 28 20:52:58.926507 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=135745) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}} May 28 20:52:59.366537 ncdv-master-1 cinder-volume[135745]: INFO dbcounter [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Registered counter for database cinder May 28 20:52:59.385683 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_db.sqlalchemy.engines [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=135745) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:52:59.385683 ncdv-master-1 cinder-volume[135745]: DEBUG dbcounter [-] [135745] Writer thread running {{(pid=135745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 20:52:59.687516 ncdv-master-1 cinder-volume[135745]: INFO cinder.rpc [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version. May 28 20:52:59.696207 ncdv-master-1 cinder-volume[135745]: INFO cinder.rpc [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version. May 28 20:52:59.700606 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_db.sqlalchemy.engines [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=135745) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:52:59.772539 ncdv-master-1 cinder-volume[135745]: INFO cinder.volume.manager [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Determined volume DB was empty at startup. May 28 20:52:59.773332 ncdv-master-1 cinder-volume[135745]: DEBUG cinder.volume.manager [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Cinder Volume DB check: vol_db_empty=True {{(pid=135745) __init__ /opt/stack/cinder/cinder/volume/manager.py:290}} May 28 20:52:59.781719 ncdv-master-1 cinder-volume[135745]: INFO cinder.volume.manager [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Service not found for updating active_backend_id, assuming default for driver init. May 28 20:52:59.782541 ncdv-master-1 cinder-volume[135745]: WARNING stevedore.named [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager May 28 20:52:59.782971 ncdv-master-1 cinder-volume[135745]: WARNING castellan.key_manager [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' May 28 20:52:59.784073 ncdv-master-1 cinder-volume[135745]: WARNING cinder.keymgr.conf_key_mgr [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] This key manager is insecure and is not recommended for production deployments: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' May 28 20:52:59.802766 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_db.sqlalchemy.engines [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=135745) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:52:59.808116 ncdv-master-1 cinder-volume[135745]: INFO cinder.rpc [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version. May 28 20:52:59.812308 ncdv-master-1 cinder-volume[135745]: INFO cinder.rpc [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Automatically selected cinder-volume RPC version 3.18 as minimum service version. May 28 20:52:59.812979 ncdv-master-1 cinder-volume[135745]: WARNING stevedore.named [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager May 28 20:52:59.813386 ncdv-master-1 cinder-volume[135745]: WARNING castellan.key_manager [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager' May 28 20:52:59.815722 ncdv-master-1 cinder-volume[135745]: INFO cinder.volume.manager [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Image-volume cache enabled for host ncdv-master-1@ceph. May 28 20:52:59.821583 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_db.sqlalchemy.engines [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=135745) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:52:59.894964 ncdv-master-1 cinder-volume[135745]: INFO oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Starting 1 workers May 28 20:52:59.900409 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Started child 135921 {{(pid=135745) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}} May 28 20:52:59.904733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Full set of CONF: {{(pid=135745) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}} May 28 20:52:59.904733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ******************************************************************************** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 28 20:52:59.904733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] Configuration options gathered from: {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 28 20:52:59.904733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 28 20:52:59.904733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 28 20:52:59.904733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ================================================================================ {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 28 20:52:59.905310 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] allow_availability_zone_fallback = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.905693 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] allow_compression_on_image_upload = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.906370 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] allowed_direct_url_schemes = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.906967 ncdv-master-1 cinder-volume[135921]: INFO cinder.service [-] Starting cinder-volume node (version 20.1.0) May 28 20:52:59.908090 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] api_rate_limit = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.908833 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] auth_strategy = keystone {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.909570 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] az_cache_duration = 3600 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.910332 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_availability_zone = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.910871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_host = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.911405 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.912086 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_stats_polling_interval = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.912769 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backup_api_class = cinder.backup.api.API {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.913293 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backup_manager = cinder.backup.manager.BackupManager {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.913882 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backup_name_template = backup-%s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.914549 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backup_use_same_host = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.915095 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backup_use_temp_snapshot = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.915912 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] chap_password = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.916507 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] chap_username = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.917051 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.917656 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_internal_tenant_project_id = 3186f9d3c2b341afa8b098ebcaaf974b {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.918213 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_internal_tenant_user_id = afc08d15f24e46e29c34d618d955bd28 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.918750 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cloned_volume_same_az = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.919267 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cluster = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.919965 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] compression_format = gzip {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.920664 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] compute_api_class = cinder.compute.nova.API {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.921271 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] config_dir = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.921782 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] config_file = ['/etc/cinder/cinder.conf'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.922333 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] config_source = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.922953 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] consistencygroup_api_class = cinder.consistencygroup.api.API {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.923515 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] control_exchange = openstack {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.924153 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] debug = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.924679 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] default_availability_zone = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.925175 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] default_group_type = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.926661 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.927225 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] default_volume_type = ceph {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.927732 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] driver_client_cert = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.928255 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] driver_client_cert_key = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.928777 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] driver_data_namespace = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.929274 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] driver_ssl_cert_path = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.929964 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] driver_ssl_cert_verify = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.930698 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] driver_use_ssl = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.931235 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] enable_force_upload = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.931713 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] enable_new_services = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.932240 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] enable_unsupported_driver = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.932901 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] enabled_backends = ['ceph'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.933419 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] enforce_multipath_for_image_xfer = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.933948 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] filter_function = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.934474 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_api_insecure = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.935111 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_api_servers = ['http://10.222.0.33/image'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.935620 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_api_ssl_compression = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.936333 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_ca_certificates_file = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.956536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_catalog_info = image:glance:publicURL {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.956536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.956536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_core_properties = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.956536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.956536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_num_retries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.956536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] glance_request_timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] goodness_function = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] graceful_shutdown_timeout = 5 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] group_api_class = cinder.group.api.API {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] host = ncdv-master-1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iet_conf = /etc/iet/ietd.conf {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_compress_on_upload = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957602 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_conversion_address_space_limit = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957602 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_conversion_cpu_limit = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957753 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Starting volume driver RBDDriver (1.2.0) May 28 20:52:59.957861 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_conversion_dir = /opt/stack/data/cinder/conversion {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957861 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_upload_use_cinder_backend = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957861 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_upload_use_internal_tenant = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.957861 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_volume_cache_enabled = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_volume_cache_max_count = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] image_volume_cache_max_size_gb = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] init_host_max_objects_retrieval = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] instance_format = [instance: %(uuid)s] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] instance_uuid_format = [instance: %(uuid)s] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iscsi_iotype = fileio {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958339 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iscsi_secondary_ip_addresses = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iscsi_target_flags = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iscsi_write_cache = on {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iser_helper = tgtadm {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iser_ip_address = 10.222.0.33 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iser_port = 3260 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.958871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_config_append = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_dir = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_file = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_options = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_rotate_interval = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_rotate_interval_type = days {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.959432 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] log_rotation_type = none {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.960095 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.960095 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.960095 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.960095 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.961155 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.961718 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] max_age = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.962329 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] max_logfile_count = 30 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.962853 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] max_logfile_size_mb = 200 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.963531 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] max_over_subscription_ratio = 20.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.964096 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] message_reap_interval = 86400 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.964590 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] message_ttl = 2592000 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.965129 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] migration_create_volume_timeout_secs = 300 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.965657 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] monkey_patch = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.966269 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] monkey_patch_modules = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.966743 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] my_ip = 10.222.0.33 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.967122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] no_snapshot_gb_quota = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.967536 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] num_iser_scan_tries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.967931 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] num_shell_tries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.968299 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] num_volume_device_scan_tries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.968655 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] nvmet_ns_id = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.969011 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] nvmet_port_id = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.969407 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_max_limit = 1000 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.969861 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_volume_ext_list = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.970258 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_volume_extension = ['cinder.api.contrib.standard_extensions'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.970647 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_volume_listen = 0.0.0.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.971243 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_volume_listen_port = 8776 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.971625 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_volume_use_ssl = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.971994 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] osapi_volume_workers = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.972352 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] per_volume_size_limit = -1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.972708 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] periodic_fuzzy_delay = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.973073 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] periodic_interval = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.973681 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] project_id_regex = [0-9a-f\-]+ {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.974113 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] public_endpoint = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.974473 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] publish_errors = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.974842 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_backup_gigabytes = 1000 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.975197 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_backups = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.975570 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_consistencygroups = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.975915 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_driver = cinder.quota.DbQuotaDriver {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.976256 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_gigabytes = 1000 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.976595 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_groups = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.976966 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_snapshots = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.977343 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] quota_volumes = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.977699 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] rate_limit_burst = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.978150 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] rate_limit_except_level = CRITICAL {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.978488 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] rate_limit_interval = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.978835 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] reinit_driver_count = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.979184 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] replication_device = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.980199 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] report_discard_supported = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.980724 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] report_interval = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.981562 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] reservation_clean_interval = 86400 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.982073 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] reservation_expire = 86400 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.982592 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] reserved_percentage = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.982994 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] resource_query_filters_file = /etc/cinder/resource_filters.json {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.983381 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] rootwrap_config = /etc/cinder/rootwrap.conf {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.983788 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] rpc_response_timeout = 120 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.984142 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] run_external_periodic_tasks = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.984527 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] scheduler_manager = cinder.scheduler.manager.SchedulerManager {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.984883 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] scst_target_driver = iscsi {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.985239 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] scst_target_iqn_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.985526 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_down_time = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.985922 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] snapshot_name_template = snapshot-%s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.986338 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] snapshot_same_host = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.986774 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] split_loggers = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.987233 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] state_path = /opt/stack/data/cinder {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.987752 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] storage_availability_zone = nova {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.988262 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] storage_protocol = iSCSI {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.988965 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] syslog_log_facility = LOG_USER {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.989353 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] target_helper = lioadm {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.989827 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] target_ip_address = 10.222.0.33 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.990233 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] target_port = 3260 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.990876 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] target_prefix = iqn.2010-10.org.openstack: {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.991444 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] target_protocol = iscsi {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.991835 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] trace_flags = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.992223 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] transfer_api_class = cinder.transfer.api.API {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.992599 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] transport_url = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.992940 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] until_refresh = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.993297 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_chap_auth = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.993655 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_default_quota_class = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.994032 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_eventlog = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.994395 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_journal = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.994733 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_json = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.995070 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_multipath_for_image_xfer = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.995411 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_stderr = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.995767 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] use_syslog = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.996185 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] verify_glance_signatures = enabled {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.996499 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_api_class = cinder.volume.api.API {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.996839 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_backend_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.997196 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_clear = zero {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.997530 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_clear_ionice = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.997908 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_clear_size = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.998308 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.998675 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_copy_bps_limit = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.999026 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_dd_blocksize = 1M {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.999374 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_manager = cinder.volume.manager.VolumeManager {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:52:59.999749 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_name_template = volume-%s {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:53:00.000105 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_service_inithost_offload = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:53:00.000576 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volume_usage_audit_period = month {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:53:00.000999 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] volumes_dir = /opt/stack/data/cinder/volumes {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:53:00.001666 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] watch_log_file = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:53:00.002724 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] zoning_mode = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 28 20:53:00.003392 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.003943 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.004599 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_concurrency.disable_process_locking = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.005372 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_concurrency.lock_path = /opt/stack/data/cinder {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.006055 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.006619 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.007176 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_metrics.metrics_process_name = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.007544 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.007896 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.008341 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.admin_password = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.008701 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.009050 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.admin_token = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.009395 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.admin_user = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.009772 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_admin_prefix = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.010158 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.010520 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_port = 35357 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.010904 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_protocol = https {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.011493 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_section = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.012040 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_type = password {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.012430 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_uri = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.012776 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.auth_version = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.013155 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.cache = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.013612 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.cafile = /opt/stack/data/ca-bundle.pem {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.014019 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.015670 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:53:00.018959 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.delay_auth_decision = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.019647 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.020498 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.http_connect_timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.021085 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.021890 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.identity_uri = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.022472 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.include_service_catalog = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.023340 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.insecure = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.024012 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.interface = public {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.024864 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.025610 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.026285 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.027537 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.028182 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.028953 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.029539 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.030178 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.030791 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.031420 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.032128 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.region_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.032727 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.033569 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.service_token_roles_required = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.034185 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.service_type = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.035032 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.token_cache_time = 300 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.035648 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.036483 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.backend = dogpile.cache.null {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.037094 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.backend_argument = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.037695 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.config_prefix = cache.oslo {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.038417 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.dead_timeout = 60.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.039274 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.debug_cache_backend = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.039815 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.enable_retry_client = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.040292 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.enable_socket_keepalive = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.040746 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.enabled = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.041234 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.expiration_time = 600 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.041684 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.hashclient_retry_attempts = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.042238 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.hashclient_retry_delay = 1.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.042820 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_dead_retry = 300 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.043620 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.044191 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.044933 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:53:00.045234 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_pool_maxsize = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.045810 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.046553 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_servers = ['localhost:11211'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.047114 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.memcache_socket_timeout = 1.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.047679 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.proxies = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.048237 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.retry_attempts = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.048994 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.retry_delay = 0.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.049568 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.socket_keepalive_count = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.050165 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.socket_keepalive_idle = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.050729 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.socket_keepalive_interval = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.051287 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.tls_allowed_ciphers = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.052055 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.tls_cafile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.052668 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.tls_certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.053347 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.tls_enabled = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.054052 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cache.tls_keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.054661 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] coordination.backend_url = etcd3+http://10.222.0.33:2379 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.055566 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.backend = sqlalchemy {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.056119 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.connection = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.056831 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.connection_debug = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.057302 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.connection_parameters = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.057815 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.connection_recycle_time = 3600 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.058330 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.connection_trace = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.059070 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.db_inc_retry_interval = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.059659 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.db_max_retries = 20 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.060247 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.db_max_retry_interval = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.060817 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.db_retry_interval = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.061569 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.max_overflow = 50 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.062273 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.max_pool_size = 5 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.062893 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.max_retries = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.063578 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.mysql_enable_ndb = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.065534 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.mysql_sql_mode = TRADITIONAL {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.066121 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.pool_timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.066818 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.retry_interval = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.067407 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.slave_connection = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.067990 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.sqlite_synchronous = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.068549 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] database.use_db_reconnect = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.070009 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.enforce_new_defaults = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.070616 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.enforce_scope = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.071328 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.policy_default_rule = default {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.071929 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.policy_dirs = ['policy.d'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.072633 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.policy_file = policy.yaml {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.073333 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.074043 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.074608 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.075420 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.075992 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.076737 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.connection_string = messaging:// {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.077234 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.enabled = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.077812 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.es_doc_type = notification {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.078454 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.es_scroll_size = 10000 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.079032 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.es_scroll_time = 2m {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.079741 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.filter_error_trace = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.080872 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.hmac_keys = SECRET_KEY {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.081330 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.sentinel_service_name = mymaster {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.081756 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.socket_timeout = 0.1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.082204 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] profiler.trace_sqlalchemy = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.082815 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] privsep_osbrick.capabilities = [21] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.083267 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] privsep_osbrick.group = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.083709 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] privsep_osbrick.helper_command = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.084132 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] privsep_osbrick.logger_name = oslo_privsep.daemon {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.084555 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.085024 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] privsep_osbrick.user = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.085685 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_sys_admin.capabilities = [0, 1, 2, 3, 12, 21] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.086122 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_sys_admin.group = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.086986 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_sys_admin.helper_command = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.087099 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_sys_admin.logger_name = oslo_privsep.daemon {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.087487 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.087938 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] cinder_sys_admin.user = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.088395 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.auth_section = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.088815 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.auth_type = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.089221 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.cafile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.089759 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.090203 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.collect_timing = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.090612 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.insecure = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.091053 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.091487 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.send_service_user_token = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.091917 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.split_loggers = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.092320 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] service_user.timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.092884 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.backend_availability_zone = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.093302 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.backend_native_threads_pool_size = 20 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.093879 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.backup_use_temp_snapshot = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.094312 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.chap_password = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.094736 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.chap_username = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.095190 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.chiscsi_conf = /etc/chelsio-iscsi/chiscsi.conf {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.095846 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.096289 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.096698 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.driver_client_cert = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.097244 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.driver_client_cert_key = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.097655 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.driver_data_namespace = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.098097 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.098506 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.098920 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.driver_use_ssl = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.099340 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.enable_deferred_deletion = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.099857 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.enable_unsupported_driver = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.100273 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.100676 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.driver [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Driver hasn't implemented _init_vendor_properties() May 28 20:53:00.100800 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.extra_capabilities = {} {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.101104 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.filter_function = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.101509 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.goodness_function = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.101714 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.driver [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:ed9cca23-4c42-4484-ab51-c6d5cb2a97f0:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=135921) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}} May 28 20:53:00.102019 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iet_conf = /etc/iet/ietd.conf {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.102439 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.102952 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.103374 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.103788 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.104200 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.104639 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iscsi_iotype = fileio {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.105080 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.105506 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iscsi_target_flags = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.106186 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iscsi_write_cache = on {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.106612 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iser_helper = tgtadm {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.107370 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iser_ip_address = 10.222.0.33 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.107810 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iser_port = 3260 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.108233 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.108665 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.109097 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.109508 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.num_shell_tries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.110048 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.110457 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.nvmet_ns_id = 10 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.110881 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.nvmet_port_id = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.111845 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.112258 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rados_connection_interval = 5 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.112775 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rados_connection_retries = 3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.113224 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_ceph_conf = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.113663 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.114193 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.114835 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.115212 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.116085 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_pool = rbd {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.116662 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_secret_uuid = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.117662 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.118049 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.rbd_user = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.118409 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.118763 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.replication_device = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.119176 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.report_discard_supported = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.119696 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.122871 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.reserved_percentage = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.123286 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.scst_target_driver = iscsi {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.123658 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.scst_target_iqn_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.123892 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_db.sqlalchemy.engines [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=135921) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 28 20:53:00.124060 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.storage_protocol = iSCSI {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.124413 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.suppress_requests_ssl_warnings = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.125099 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.target_helper = tgtadm {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.125603 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.target_ip_address = 10.222.0.33 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.126612 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.target_port = 3260 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.126612 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.127522 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.target_protocol = iscsi {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.127889 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.trace_flags = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.128269 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.unique_fqdn_network = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.128628 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.use_chap_auth = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.128985 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.129334 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_backend_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.129701 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_clear = zero {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.130412 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_clear_ionice = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.130778 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_clear_size = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.131028 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writer thread running {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 28 20:53:00.131128 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.131489 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.132075 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.132454 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.133027 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] backend_defaults.volumes_dir = /opt/stack/data/cinder/volumes {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.133435 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_reports.file_event_handler = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.133831 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.134218 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_reports.log_dir = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.134606 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.backend_availability_zone = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.134956 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.backend_host = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.135309 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.backend_native_threads_pool_size = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.135657 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.backup_use_temp_snapshot = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.136008 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.chap_password = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.136565 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.chap_username = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.136925 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.chiscsi_conf = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.137288 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.deferred_deletion_delay = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.137618 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.deferred_deletion_purge_interval = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.137999 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.driver_client_cert = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.138353 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.driver_client_cert_key = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.138700 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.driver_data_namespace = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.139045 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.driver_ssl_cert_path = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.139443 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.driver_ssl_cert_verify = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.139843 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.driver_use_ssl = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.140356 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.enable_deferred_deletion = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.140806 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.enable_unsupported_driver = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.141245 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.enforce_multipath_for_image_xfer = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.141687 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.extra_capabilities = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.142154 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.filter_function = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.142638 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.goodness_function = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.143173 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iet_conf = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.143541 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.image_upload_use_cinder_backend = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.143891 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.image_upload_use_internal_tenant = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.144257 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.image_volume_cache_enabled = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.144603 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.image_volume_cache_max_count = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.144949 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.image_volume_cache_max_size_gb = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.145298 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iscsi_iotype = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.145646 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iscsi_secondary_ip_addresses = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.146042 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iscsi_target_flags = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.146587 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iscsi_write_cache = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.146994 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iser_helper = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.147346 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iser_ip_address = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.147694 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iser_port = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.148036 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.iser_target_prefix = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.148381 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.max_over_subscription_ratio = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.148728 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.num_iser_scan_tries = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.149283 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.num_shell_tries = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.149659 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.num_volume_device_scan_tries = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.150026 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.nvmet_ns_id = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.150471 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.nvmet_port_id = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.150862 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rados_connect_timeout = -1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.151194 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rados_connection_interval = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.151590 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rados_connection_retries = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.152040 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_ceph_conf = /etc/ceph/ceph.conf {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.152393 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_cluster_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.152737 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_exclusive_cinder_pool = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.153178 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_flatten_volume_from_snapshot = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.153547 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_max_clone_depth = 5 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.153926 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_pool = volumes {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.154286 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_secret_uuid = 09c9249f-256c-45da-b5e8-aced76ddcb19 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.154641 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_store_chunk_size = 4 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.154996 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.rbd_user = cinder {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.155337 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.replication_connect_timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.155680 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.replication_device = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.156020 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.report_discard_supported = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.156369 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.report_dynamic_total_capacity = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.156887 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.reserved_percentage = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.157229 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.scst_target_driver = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.157569 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.scst_target_iqn_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.158030 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.storage_protocol = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.158421 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.suppress_requests_ssl_warnings = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.158765 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.target_helper = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.159111 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.target_ip_address = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.159557 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.target_port = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.159903 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.target_prefix = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.160247 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.target_protocol = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.160692 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.trace_flags = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.161041 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.unique_fqdn_network = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.161506 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.use_chap_auth = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.161870 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.use_multipath_for_image_xfer = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.162236 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_backend_name = ceph {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.162576 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_clear = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.162916 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_clear_ionice = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.163260 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_clear_size = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.163627 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_copy_blkio_cgroup_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.163991 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_copy_bps_limit = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.164329 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_dd_blocksize = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.164687 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volume_driver = cinder.volume.drivers.rbd.RBDDriver {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.165051 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ceph.volumes_dir = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.165635 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.166025 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.166412 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.166944 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.167333 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.167684 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.168060 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.168408 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.168757 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.169114 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.169582 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.169966 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.170373 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.170825 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.171268 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.171670 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.172021 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.172372 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.172725 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.173076 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.173450 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.174362 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.174768 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.ssl = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.175138 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.ssl_ca_file = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.175499 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.ssl_cert_file = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.175847 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.176208 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.ssl_key_file = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.176591 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_rabbit.ssl_version = {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.177087 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_notifications.driver = [] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.177461 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_notifications.retry = -1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.177889 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.178240 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] oslo_messaging_notifications.transport_url = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.178586 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] key_manager.backend = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.178872 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] key_manager.fixed_key = **** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.179316 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.auth_endpoint = http://localhost/identity/v3 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.179756 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.barbican_api_version = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.180139 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.barbican_endpoint = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.180528 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.barbican_endpoint_type = public {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.180970 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.barbican_region_name = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.181409 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.cafile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.183612 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.184007 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.collect_timing = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.184365 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.insecure = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.184723 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.185099 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.number_of_retries = 60 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.185469 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.retry_delay = 1 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.185825 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.send_service_user_token = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.186216 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.split_loggers = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.186567 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.187152 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.verify_ssl = True {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.187459 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican.verify_ssl_path = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.187822 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.auth_section = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.188175 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.auth_type = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.188523 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.cafile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.188862 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.189208 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.collect_timing = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.190228 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.insecure = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.190576 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.191024 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.split_loggers = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.191371 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] barbican_service_user.timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.191758 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.approle_role_id = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.192101 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.approle_secret_id = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.192441 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.cafile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.192795 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.certfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.193144 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.collect_timing = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.193492 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.insecure = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.193880 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.keyfile = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.194227 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.kv_mountpoint = secret {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.194600 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.kv_version = 2 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.194937 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.namespace = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.195282 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.root_token_id = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.195628 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.split_loggers = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.195969 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.ssl_ca_crt_file = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.196309 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.timeout = None {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.196660 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.use_ssl = False {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.197202 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] vault.vault_url = http://127.0.0.1:8200 {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 28 20:53:00.197492 ncdv-master-1 cinder-volume[135745]: DEBUG oslo_service.service [None req-f408706c-a666-4ad6-84e7-267d7bcb7de1 None None] ******************************************************************************** {{(pid=135745) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 28 20:53:00.203375 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.db.sqlalchemy.api [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Building query based on filter {{(pid=135921) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3846}} May 28 20:53:00.225490 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Configuration image_conversion_dir is None or the path doesn't exist. {{(pid=135921) cleanup_temporary_file /opt/stack/cinder/cinder/image/image_utils.py:955}} May 28 20:53:00.233769 ncdv-master-1 cinder-volume[135921]: INFO cinder.keymgr.migration [None req-e8341fe8-37db-4f46-8347-466526f4eb40 None None] Not migrating encryption keys because the ConfKeyManager is still in use. May 28 20:53:00.242716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:53:00.281507 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:53:00.316998 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 20:53:00.347807 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Driver initialization completed successfully. May 28 20:53:00.348582 ncdv-master-1 cinder-volume[135921]: INFO cinder.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Initiating service 3 cleanup May 28 20:53:00.360340 ncdv-master-1 cinder-volume[135921]: INFO cinder.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Service 3 cleanup completed. May 28 20:53:00.360810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.service [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Creating RPC server for service cinder-volume {{(pid=135921) start /opt/stack/cinder/cinder/service.py:231}} May 28 20:53:00.369211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.service [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=135921) start /opt/stack/cinder/cinder/service.py:237}} May 28 20:53:00.413626 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Initializing RPC dependent components of volume driver RBDDriver (1.2.0) May 28 20:53:00.415130 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:53:00.441654 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:53:00.540913 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8c7bf7f5-f21d-4d2c-abbb-8cd1b772ba3e None None] Driver post RPC initialization completed successfully. May 28 20:53:09.834753 ncdv-master-1 cinder-volume[135745]: DEBUG dbcounter [-] [135745] Writing DB stats cinder:SELECT=16,cinder:INSERT=1 {{(pid=135745) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:10.486293 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:10.489218 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=23,cinder:INSERT=1,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:40.556145 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:53:40.557681 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:00.540638 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 20:54:00.541411 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:54:00.563322 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:54:00.587128 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 20:54:00.590881 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:00.592103 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:30.589581 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:54:40.594860 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:55:00.537568 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 20:55:00.538317 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:55:00.560450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:55:00.583894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 20:55:00.594236 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:55:10.606973 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:55:30.607408 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:55:30.616937 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:55:50.613477 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=1,cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:56:10.617628 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:56:30.624139 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:00.539507 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 20:57:00.540195 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:57:00.563628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:57:00.585508 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 20:57:00.626033 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:10.632669 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:40.630088 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:57:50.628426 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:58:00.540440 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 20:58:00.541085 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:58:00.562187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:58:00.584137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 20:58:10.632101 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:58:30.639603 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:58:30.641403 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:58:50.638363 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:59:00.539364 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 20:59:00.540001 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:59:00.561927 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 20:59:00.582894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 20:59:10.639161 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:59:30.637080 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 20:59:40.643613 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:00:00.537407 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:00:00.538090 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:00:00.558954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:00:00.580600 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:00:10.644382 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:00:10.645884 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:00:30.652781 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:01:00.536004 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:01:00.536646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:01:00.557705 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:01:00.578625 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:01:00.655827 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:01:10.653363 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:01:30.652051 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:01:50.650818 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:00.533906 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:02:00.534712 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:02:00.576441 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:02:00.612448 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:02:10.653797 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:30.651113 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:02:50.657989 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:00.539479 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:03:00.540249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:03:00.628944 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:03:00.717191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:03:00.724275 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:10.742226 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:30.733109 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:50.731358 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:03:50.732116 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:00.535794 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:04:00.536649 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:04:00.587973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:04:00.648246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:04:20.741298 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:30.735118 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:40.744239 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:04:50.757494 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:00.537583 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:05:00.545872 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:05:00.652028 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:05:00.762081 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:05:00.773227 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:28.672048 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:05:28.750105 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Flow 'volume_create_manager' (4487cb1f-e188-4506-9c5f-c4a61cbd56e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:05:28.761501 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (66500fee-b676-400a-b986-9a9654555b7a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:05:28.811455 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=10,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:28.842240 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (66500fee-b676-400a-b986-9a9654555b7a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:05:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveAutoBlockMigrationV225Test-volume-1492805112',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=96682203-c2c9-4aeb-9901-07faacbd380d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='45a2b4e7e1484ea3807bc8d8bc31296c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:05:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:05:29Z,use_quota=True,user_id='fc6ee6dba5d24e90bff48384539fcd7f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:05:28.847755 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bcd7f2ee-ba35-4386-9f34-5cc45dd0fe29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:05:28.853713 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (bcd7f2ee-ba35-4386-9f34-5cc45dd0fe29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:05:28.862287 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e78542fa-20d5-44af-ad67-f98946e96bbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:05:28.869381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e78542fa-20d5-44af-ad67-f98946e96bbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '96682203-c2c9-4aeb-9901-07faacbd380d', 'volume_name': 'volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:05:28.882464 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8f62500-af29-45cb-b79b-97f4ffd7ae52) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:05:28.891040 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c8f62500-af29-45cb-b79b-97f4ffd7ae52) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:05:28.902405 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c6a96f9e-e939-4db0-a569-46677888cadd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:05:28.904094 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume 96682203-c2c9-4aeb-9901-07faacbd380d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'volume_size': 1} May 28 21:05:28.904481 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] creating volume 'volume-96682203-c2c9-4aeb-9901-07faacbd380d' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:05:28.905809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:05:29.899933 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:05:29.903654 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:05:29.908376 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c6a96f9e-e939-4db0-a569-46677888cadd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:05:29.927447 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ccf7fbd-9c3f-478b-b42f-aaae374b750b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:05:30.632738 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-96682203-c2c9-4aeb-9901-07faacbd380d (96682203-c2c9-4aeb-9901-07faacbd380d): created successfully May 28 21:05:30.637100 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ccf7fbd-9c3f-478b-b42f-aaae374b750b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:05:30.645131 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Flow 'volume_create_manager' (4487cb1f-e188-4506-9c5f-c4a61cbd56e2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:05:30.728714 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e297ca13-98f6-4536-ad35-925b3443e951 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Created volume successfully. May 28 21:05:33.911962 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d9ae8113-5013-42e8-a3f1-514606707e41 req-0659f77d-022e-4817-8f6d-423b354051b6 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:05:34.845832 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d9ae8113-5013-42e8-a3f1-514606707e41 req-0659f77d-022e-4817-8f6d-423b354051b6 tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.934s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:05:34.847444 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-d9ae8113-5013-42e8-a3f1-514606707e41 req-0659f77d-022e-4817-8f6d-423b354051b6 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '96682203-c2c9-4aeb-9901-07faacbd380d', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:05:35.100854 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-d9ae8113-5013-42e8-a3f1-514606707e41 req-0659f77d-022e-4817-8f6d-423b354051b6 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Connection info returned from driver {'name': 'volumes/volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '96682203-c2c9-4aeb-9901-07faacbd380d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6b532fc1-e9fc-4d98-a01b-f4d77908d4b4'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:05:35.308302 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-d9ae8113-5013-42e8-a3f1-514606707e41 req-0659f77d-022e-4817-8f6d-423b354051b6 tempest-LiveAutoBlockMigrationV225Test-605147657 None] attachment_update completed successfully. May 28 21:05:59.709619 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=16,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:59.710404 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=23,cinder:UPDATE=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:05:59.718390 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-d6e0cd16-0bf1-422e-8920-a0626d28b0b1 tempest-LiveAutoBlockMigrationV225Test-664987053 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:00.542342 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:06:00.543808 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:06:00.614925 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:06:00.667608 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:06:00.689051 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-d6e0cd16-0bf1-422e-8920-a0626d28b0b1 tempest-LiveAutoBlockMigrationV225Test-664987053 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.972s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:00.689895 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-d6e0cd16-0bf1-422e-8920-a0626d28b0b1 tempest-LiveAutoBlockMigrationV225Test-664987053 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '96682203-c2c9-4aeb-9901-07faacbd380d', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:06:00.799717 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-d6e0cd16-0bf1-422e-8920-a0626d28b0b1 tempest-LiveAutoBlockMigrationV225Test-664987053 None] Connection info returned from driver {'name': 'volumes/volume-96682203-c2c9-4aeb-9901-07faacbd380d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '96682203-c2c9-4aeb-9901-07faacbd380d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c9bc643b-bfe6-4cd1-a73e-2678ba9a8f79'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:06:00.951490 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-d6e0cd16-0bf1-422e-8920-a0626d28b0b1 tempest-LiveAutoBlockMigrationV225Test-664987053 None] attachment_update completed successfully. May 28 21:06:15.363398 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-8d75545b-2dc7-4c88-a409-da108afb2eeb tempest-LiveAutoBlockMigrationV225Test-664987053 None] Terminate volume connection completed successfully. May 28 21:06:15.364120 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9c9f81b7-8018-41b4-971c-e075172d8f2f req-8d75545b-2dc7-4c88-a409-da108afb2eeb tempest-LiveAutoBlockMigrationV225Test-664987053 None] Deleting attachment 6b532fc1-e9fc-4d98-a01b-f4d77908d4b4. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:06:23.250604 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-3bf46771-7e11-4f56-9f44-aa08a136add0 req-7ac563cb-a555-494d-99ee-ee4a333fda92 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Terminate volume connection completed successfully. May 28 21:06:23.251745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-3bf46771-7e11-4f56-9f44-aa08a136add0 req-7ac563cb-a555-494d-99ee-ee4a333fda92 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Deleting attachment c9bc643b-bfe6-4cd1-a73e-2678ba9a8f79. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:06:23.998926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:06:24.037171 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Flow 'volume_create_manager' (097acf87-2811-449d-b4e6-3f0ddbbd5499) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:06:24.046469 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f0995835-9d34-4db6-9920-9df4e9fc9e23) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:24.068290 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f0995835-9d34-4db6-9920-9df4e9fc9e23) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:06:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveMigrationTest-volume-1415183702',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b107b45a-f2d1-4795-8559-3836162f6660,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2f5c3054461d4d07861a1685cfeba36a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:06:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:06:24Z,use_quota=True,user_id='7b44b524592e4a57a26681767b9c4541',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:24.074372 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8d632d9c-9991-40e0-9794-7e89434b5fb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:24.088245 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8d632d9c-9991-40e0-9794-7e89434b5fb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:24.101408 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (12bd0729-b109-4718-9953-f7d280c99099) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:24.106285 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (12bd0729-b109-4718-9953-f7d280c99099) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b107b45a-f2d1-4795-8559-3836162f6660', 'volume_name': 'volume-b107b45a-f2d1-4795-8559-3836162f6660', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:24.111845 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ee8fc2c5-0bc1-470d-8463-675a86f27fe7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:24.117641 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ee8fc2c5-0bc1-470d-8463-675a86f27fe7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:24.123075 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d7fed5d5-1e19-46dc-a1c8-69a4e8513929) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:24.124181 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Volume b107b45a-f2d1-4795-8559-3836162f6660: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b107b45a-f2d1-4795-8559-3836162f6660', 'volume_size': 1} May 28 21:06:24.124583 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] creating volume 'volume-b107b45a-f2d1-4795-8559-3836162f6660' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:06:24.125248 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:06:24.663240 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:06:24.666920 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:06:24.668581 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d7fed5d5-1e19-46dc-a1c8-69a4e8513929) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b107b45a-f2d1-4795-8559-3836162f6660', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:24.675890 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ccff8f00-784c-4d1f-9af5-1eb3af6a2a9a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:24.766604 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Volume volume-b107b45a-f2d1-4795-8559-3836162f6660 (b107b45a-f2d1-4795-8559-3836162f6660): created successfully May 28 21:06:24.778165 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ccff8f00-784c-4d1f-9af5-1eb3af6a2a9a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:24.803863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Flow 'volume_create_manager' (097acf87-2811-449d-b4e6-3f0ddbbd5499) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:06:24.911412 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-07df34e7-cc6f-4186-a9ab-d48e4326a448 tempest-LiveMigrationTest-1347342374 None] Created volume successfully. May 28 21:06:25.180716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:06:25.219118 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Flow 'volume_create_manager' (ede803e5-67a6-4df9-82fb-b5089984e324) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:06:25.225798 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c8db109-3f9d-4fa2-9e95-8044a1575dd8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:25.264840 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c8db109-3f9d-4fa2-9e95-8044a1575dd8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:06:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-795747646',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bdc702a5-c0fa-423b-b6bf-14c735b9af34,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='45a2b4e7e1484ea3807bc8d8bc31296c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:06:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:06:25Z,use_quota=True,user_id='fc6ee6dba5d24e90bff48384539fcd7f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:25.271222 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c66f5c5a-a906-4fab-bb42-000cbb2b287d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:25.287196 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c66f5c5a-a906-4fab-bb42-000cbb2b287d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:25.292555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c2c54d90-8732-4e4d-ab8a-eaed17e2cca5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:25.413875 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c2c54d90-8732-4e4d-ab8a-eaed17e2cca5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'volume_name': 'volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:25.419593 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a44573a4-f887-48a6-a799-d42706e45137) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:25.424210 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a44573a4-f887-48a6-a799-d42706e45137) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:25.429561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b451a8ce-67e0-4d11-8439-d2d6459195d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:25.433919 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume bdc702a5-c0fa-423b-b6bf-14c735b9af34: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:06:25.433919 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Cloning bdc702a5-c0fa-423b-b6bf-14c735b9af34 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:06:25.441970 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:06:25.451055 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Preparing cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:881}} May 28 21:06:25.452600 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:06:26.178460 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-483c2d28-c6ec-4281-bd03-4fcbe61900a2 req-6545e6cd-0479-4683-b0ef-f69da868a08e tempest-LiveMigrationTest-1347342374 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:27.095884 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-483c2d28-c6ec-4281-bd03-4fcbe61900a2 req-6545e6cd-0479-4683-b0ef-f69da868a08e tempest-LiveMigrationTest-1347342374 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.917s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:27.097225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-483c2d28-c6ec-4281-bd03-4fcbe61900a2 req-6545e6cd-0479-4683-b0ef-f69da868a08e tempest-LiveMigrationTest-1347342374 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b107b45a-f2d1-4795-8559-3836162f6660', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'b107b45a-f2d1-4795-8559-3836162f6660', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:06:27.185752 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-483c2d28-c6ec-4281-bd03-4fcbe61900a2 req-6545e6cd-0479-4683-b0ef-f69da868a08e tempest-LiveMigrationTest-1347342374 None] Connection info returned from driver {'name': 'volumes/volume-b107b45a-f2d1-4795-8559-3836162f6660', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b107b45a-f2d1-4795-8559-3836162f6660', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'bb362e70-84ad-4ddf-92f4-6ca598113271'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:06:27.342513 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-483c2d28-c6ec-4281-bd03-4fcbe61900a2 req-6545e6cd-0479-4683-b0ef-f69da868a08e tempest-LiveMigrationTest-1347342374 None] attachment_update completed successfully. May 28 21:06:30.802127 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=48,cinder:UPDATE=9 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:38.220973 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=32,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:06:38.224556 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-30afda66-414b-4254-84b1-4414c5b85c37 req-2a2cccac-e3da-44b2-9497-a0ff772089e7 tempest-LiveMigrationTest-1002282877 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:38.961527 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-30afda66-414b-4254-84b1-4414c5b85c37 req-2a2cccac-e3da-44b2-9497-a0ff772089e7 tempest-LiveMigrationTest-1002282877 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.737s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:38.962359 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-30afda66-414b-4254-84b1-4414c5b85c37 req-2a2cccac-e3da-44b2-9497-a0ff772089e7 tempest-LiveMigrationTest-1002282877 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b107b45a-f2d1-4795-8559-3836162f6660', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'b107b45a-f2d1-4795-8559-3836162f6660', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:06:39.100629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-30afda66-414b-4254-84b1-4414c5b85c37 req-2a2cccac-e3da-44b2-9497-a0ff772089e7 tempest-LiveMigrationTest-1002282877 None] Connection info returned from driver {'name': 'volumes/volume-b107b45a-f2d1-4795-8559-3836162f6660', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b107b45a-f2d1-4795-8559-3836162f6660', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0369a8ce-67ca-4e60-96fe-3f716c938e15'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:06:39.260444 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-30afda66-414b-4254-84b1-4414c5b85c37 req-2a2cccac-e3da-44b2-9497-a0ff772089e7 tempest-LiveMigrationTest-1002282877 None] attachment_update completed successfully. May 28 21:06:42.812775 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Image fetch details: dest /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph, size 616.00 MB, duration 17.35 sec {{(pid=135921) fetch /opt/stack/cinder/cinder/image/image_utils.py:580}} May 28 21:06:42.813459 ncdv-master-1 cinder-volume[135921]: INFO cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Image download 616.00 MB at 35.50 MB/s May 28 21:06:42.849796 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:42.976493 ncdv-master-1 sudo[146543]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph May 28 21:06:42.977851 ncdv-master-1 sudo[146543]: pam_unix(sudo:session): session opened for user root by (uid=0) May 28 21:06:44.985035 ncdv-master-1 sudo[146543]: pam_unix(sudo:session): session closed for user root May 28 21:06:44.988290 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph" returned: 0 in 2.137s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:44.998934 ncdv-master-1 cinder-volume[135921]: WARNING py.warnings [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 28 21:06:44.998934 ncdv-master-1 cinder-volume[135921]: debtcollector.deprecate( May 28 21:06:44.998934 ncdv-master-1 cinder-volume[135921]: May 28 21:06:44.998934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Temporary image 9586d0d3-c540-4636-bb36-06bb4b16c195 is fetched for user fc6ee6dba5d24e90bff48384539fcd7f. {{(pid=135921) fetch /opt/stack/cinder/cinder/image/image_utils.py:1088}} May 28 21:06:45.203305 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:45.315635 ncdv-master-1 sudo[146568]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph May 28 21:06:45.316421 ncdv-master-1 sudo[146568]: pam_unix(sudo:session): session opened for user root by (uid=0) May 28 21:06:46.541023 ncdv-master-1 sudo[146568]: pam_unix(sudo:session): session closed for user root May 28 21:06:46.545275 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph" returned: 0 in 1.340s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:46.545275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] creating volume 'volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:06:46.552438 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:06:47.107605 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:06:47.108225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:06:47.259005 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Attempting download of 9586d0d3-c540-4636-bb36-06bb4b16c195 ((None, None)) to volume bdc702a5-c0fa-423b-b6bf-14c735b9af34. {{(pid=135921) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1181}} May 28 21:06:47.314835 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_9586d0d3-c540-4636-bb36-06bb4b16c195_n8f_k66l {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:47.359776 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:06:47.384175 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Flow 'volume_create_manager' (0d1890d5-8b63-4c20-8b55-0459637a4f10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:06:47.391434 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8e0dc06a-50ce-4559-81c5-8b89df3513b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:47.411308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8e0dc06a-50ce-4559-81c5-8b89df3513b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:06:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachSCSIVolumeTestJSON-volume-163136077',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b87de5c2-3b39-474a-950a-0a50a6da60be,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b059b8b9ea6e4afbb5d034c9d1a036d2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:06:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:06:47Z,use_quota=True,user_id='77a6068e4dc34fb6b8d1447472692e55',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:47.416760 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (05557c58-479e-4e1d-94bc-ca52ddd86c2f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:47.421880 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (05557c58-479e-4e1d-94bc-ca52ddd86c2f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:47.427123 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4a5959e3-2235-4a6b-9010-ad382a67719c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:47.432023 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4a5959e3-2235-4a6b-9010-ad382a67719c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b87de5c2-3b39-474a-950a-0a50a6da60be', 'volume_name': 'volume-b87de5c2-3b39-474a-950a-0a50a6da60be', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:47.437678 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (237ab714-f3fb-4919-bfc6-dcabf6e159a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:47.442776 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (237ab714-f3fb-4919-bfc6-dcabf6e159a6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:47.444696 ncdv-master-1 sudo[146591]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_9586d0d3-c540-4636-bb36-06bb4b16c195_n8f_k66l May 28 21:06:47.446075 ncdv-master-1 sudo[146591]: pam_unix(sudo:session): session opened for user root by (uid=0) May 28 21:06:47.447809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8d03e581-2ffa-42a8-8910-3ef7def74f2f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:47.449292 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Volume b87de5c2-3b39-474a-950a-0a50a6da60be: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b87de5c2-3b39-474a-950a-0a50a6da60be', 'volume_size': 1} May 28 21:06:47.449956 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] creating volume 'volume-b87de5c2-3b39-474a-950a-0a50a6da60be' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:06:47.451222 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:06:47.994143 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:06:47.995418 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:06:47.999309 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8d03e581-2ffa-42a8-8910-3ef7def74f2f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b87de5c2-3b39-474a-950a-0a50a6da60be', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:48.009623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd50bd30-f5d1-4614-a1aa-26601ab02e0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:06:48.092955 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Volume volume-b87de5c2-3b39-474a-950a-0a50a6da60be (b87de5c2-3b39-474a-950a-0a50a6da60be): created successfully May 28 21:06:48.097353 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fd50bd30-f5d1-4614-a1aa-26601ab02e0a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:06:48.103104 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Flow 'volume_create_manager' (0d1890d5-8b63-4c20-8b55-0459637a4f10) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:06:48.223824 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8422f473-5fe7-4dfa-b6be-fedea2d4f4f0 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Created volume successfully. May 28 21:06:48.560212 ncdv-master-1 sudo[146591]: pam_unix(sudo:session): session closed for user root May 28 21:06:48.562215 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_9586d0d3-c540-4636-bb36-06bb4b16c195_n8f_k66l" returned: 0 in 1.247s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:48.562829 ncdv-master-1 cinder-volume[135921]: WARNING py.warnings [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 28 21:06:48.562829 ncdv-master-1 cinder-volume[135921]: debtcollector.deprecate( May 28 21:06:48.562829 ncdv-master-1 cinder-volume[135921]: May 28 21:06:48.563775 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:48.660849 ncdv-master-1 sudo[146611]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph May 28 21:06:48.662440 ncdv-master-1 sudo[146611]: pam_unix(sudo:session): session opened for user root by (uid=0) May 28 21:06:49.921973 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-60902b66-1a89-40b0-b2b8-927942c73e4a req-0a145ae5-2bc5-4964-9404-34d9c8b68928 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:50.000378 ncdv-master-1 sudo[146611]: pam_unix(sudo:session): session closed for user root May 28 21:06:50.002949 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph" returned: 0 in 1.439s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:50.003977 ncdv-master-1 cinder-volume[135921]: WARNING py.warnings [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 28 21:06:50.003977 ncdv-master-1 cinder-volume[135921]: debtcollector.deprecate( May 28 21:06:50.003977 ncdv-master-1 cinder-volume[135921]: May 28 21:06:50.005232 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] 9586d0d3-c540-4636-bb36-06bb4b16c195 was vhdx, converting to raw {{(pid=135921) fetch_to_volume_format /opt/stack/cinder/cinder/image/image_utils.py:793}} May 28 21:06:50.006576 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpysoab3df {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:50.033000 ncdv-master-1 sudo[146618]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpysoab3df May 28 21:06:50.035229 ncdv-master-1 sudo[146618]: pam_unix(sudo:session): session opened for user root by (uid=0) May 28 21:06:50.750902 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-60902b66-1a89-40b0-b2b8-927942c73e4a req-0a145ae5-2bc5-4964-9404-34d9c8b68928 tempest-AttachSCSIVolumeTestJSON-1219301458 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.829s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:50.752282 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-60902b66-1a89-40b0-b2b8-927942c73e4a req-0a145ae5-2bc5-4964-9404-34d9c8b68928 tempest-AttachSCSIVolumeTestJSON-1219301458 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b87de5c2-3b39-474a-950a-0a50a6da60be', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'b87de5c2-3b39-474a-950a-0a50a6da60be', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:06:50.830116 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-60902b66-1a89-40b0-b2b8-927942c73e4a req-0a145ae5-2bc5-4964-9404-34d9c8b68928 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Connection info returned from driver {'name': 'volumes/volume-b87de5c2-3b39-474a-950a-0a50a6da60be', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b87de5c2-3b39-474a-950a-0a50a6da60be', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e22c3da3-d069-4754-b4a1-122c5e663737'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:06:51.006243 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-60902b66-1a89-40b0-b2b8-927942c73e4a req-0a145ae5-2bc5-4964-9404-34d9c8b68928 tempest-AttachSCSIVolumeTestJSON-1219301458 None] attachment_update completed successfully. May 28 21:06:52.343570 ncdv-master-1 sudo[146618]: pam_unix(sudo:session): session closed for user root May 28 21:06:52.346033 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpysoab3df" returned: 0 in 2.339s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:52.347144 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:06:52.458825 ncdv-master-1 sudo[146656]: ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph May 28 21:06:52.459613 ncdv-master-1 sudo[146656]: pam_unix(sudo:session): session opened for user root by (uid=0) May 28 21:06:52.918676 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-30afda66-414b-4254-84b1-4414c5b85c37 req-87616e09-2f70-4d37-91d5-036dc4f0d72a tempest-LiveMigrationTest-1002282877 None] Terminate volume connection completed successfully. May 28 21:06:52.919431 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-30afda66-414b-4254-84b1-4414c5b85c37 req-87616e09-2f70-4d37-91d5-036dc4f0d72a tempest-LiveMigrationTest-1002282877 None] Deleting attachment bb362e70-84ad-4ddf-92f4-6ca598113271. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:06:53.901384 ncdv-master-1 sudo[146656]: pam_unix(sudo:session): session closed for user root May 28 21:06:53.904385 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph" returned: 0 in 1.556s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:06:53.905071 ncdv-master-1 cinder-volume[135921]: WARNING py.warnings [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena' May 28 21:06:53.905071 ncdv-master-1 cinder-volume[135921]: debtcollector.deprecate( May 28 21:06:53.905071 ncdv-master-1 cinder-volume[135921]: May 28 21:06:53.905652 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Image conversion details: src /opt/stack/data/cinder/conversion/image_fetch_9586d0d3-c540-4636-bb36-06bb4b16c195_vweychvqncdv-master-1@ceph, size 700.00 MB, duration 2.34 sec, destination /opt/stack/data/cinder/conversion/tmpysoab3df {{(pid=135921) _convert_image /opt/stack/cinder/cinder/image/image_utils.py:412}} May 28 21:06:53.906101 ncdv-master-1 cinder-volume[135921]: INFO cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Converted 700.00 MB image at 299.10 MB/s May 28 21:06:53.908245 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:06:54.206682 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:06:54.212307 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:06:54.340073 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] deleting rbd volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:06:55.981991 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-0ca69edc-c752-4d97-9c48-ba70e58d5813 req-96875afc-dfd0-4a07-9bd4-f44b7492e736 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Terminate volume connection completed successfully. May 28 21:06:55.984186 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-0ca69edc-c752-4d97-9c48-ba70e58d5813 req-96875afc-dfd0-4a07-9bd4-f44b7492e736 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Deleting attachment e22c3da3-d069-4754-b4a1-122c5e663737. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:07:00.539827 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:07:00.541355 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:00.653098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:00.732246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:07:00.746481 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpysoab3df volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:07:03.913493 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=49,cinder:UPDATE=8,cinder:INSERT=1,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:03.933141 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-6f891289-95e9-4044-bd81-56f333deec53 req-bb37ef4b-d295-447b-94b1-d0ed1aa81ccf tempest-LiveMigrationTest-1347342374 None] Terminate volume connection completed successfully. May 28 21:07:03.933804 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-6f891289-95e9-4044-bd81-56f333deec53 req-bb37ef4b-d295-447b-94b1-d0ed1aa81ccf tempest-LiveMigrationTest-1347342374 None] Deleting attachment 0369a8ce-67ca-4e60-96fe-3f716c938e15. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:07:07.573743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:07:07.605412 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Flow 'volume_create_manager' (6196fd31-ca4a-4b0e-8fac-ad3a3ed8171b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:07.613944 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5e8ee26e-1eed-4baf-8d11-dbc579d4d5c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:07.648181 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5e8ee26e-1eed-4baf-8d11-dbc579d4d5c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:07:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-824290313',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=69bfe0ab-8925-40f5-bc39-33e09bf78a5e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2f5c3054461d4d07861a1685cfeba36a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:07:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:07:07Z,use_quota=True,user_id='7b44b524592e4a57a26681767b9c4541',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:07.655723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2065fcbb-1f4b-429b-a7f2-f0ecd15bacc2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:07.661597 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2065fcbb-1f4b-429b-a7f2-f0ecd15bacc2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:07.670889 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (df5ae535-12d6-4561-810f-d11115210c00) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:07.802731 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (df5ae535-12d6-4561-810f-d11115210c00) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'volume_name': 'volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:07.809388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d302669-3925-4f1f-b21c-c372cf21cd8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:07.814294 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0d302669-3925-4f1f-b21c-c372cf21cd8e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:07.820202 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6dd4b542-e323-4966-8d6d-bd348b44554a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:07.821475 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Volume 69bfe0ab-8925-40f5-bc39-33e09bf78a5e: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:07:07.822000 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Cloning 69bfe0ab-8925-40f5-bc39-33e09bf78a5e from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:07:10.195545 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Lock "b'cinder-b87de5c2-3b39-474a-950a-0a50a6da60be-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:07:10.196501 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=35,cinder:UPDATE=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:10.294956 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:10.774354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:07:10.775162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Volume volume-b87de5c2-3b39-474a-950a-0a50a6da60be is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:07:11.132363 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] deleting rbd volume volume-b87de5c2-3b39-474a-950a-0a50a6da60be {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:07:22.620883 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota. May 28 21:07:22.622494 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota. May 28 21:07:22.749443 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Created reservations ['34eb6183-1400-414c-a8a0-1b8307da82d9', '43dfbfea-f795-4b89-a9f7-311a1b96ae5e', '051e96b1-7605-43b8-83a8-e6005c883776', '6dd995a3-180d-4f76-9b8a-a9dd50a635fe'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:07:23.024294 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:23.052892 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:23.102678 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:07:23.112015 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Deleted volume successfully. May 28 21:07:23.220749 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-56abc195-0e6e-4d3a-977b-8fdf93e1af79 tempest-AttachSCSIVolumeTestJSON-1219301458 None] Lock "b'cinder-b87de5c2-3b39-474a-950a-0a50a6da60be-delete_volume'" released by "delete_volume" :: held 13.025s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:07:33.118332 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=18,cinder:UPDATE=9,cinder:INSERT=1,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:41.340044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:07:41.398267 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Flow 'volume_create_manager' (5e3a7ea8-d502-409b-a1a0-4063bea8c699) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:41.412760 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4a7444e4-cf74-4ac1-9ea6-6c27cde42146) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:41.510507 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4a7444e4-cf74-4ac1-9ea6-6c27cde42146) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:07:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-537731882',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=497e0018-ccb8-45cf-b799-e98400f1ef6f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f7a9c4408ec1432cb05c5256657886d4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:07:41Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:07:41Z,use_quota=True,user_id='be94d10c650645658c755b7fc1595370',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:41.525930 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bd72f89-7808-481c-bcd3-99884885e684) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:41.537219 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3bd72f89-7808-481c-bcd3-99884885e684) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:41.558318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (22454c82-c23c-4081-8eb5-a6630334964a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:41.565135 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (22454c82-c23c-4081-8eb5-a6630334964a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '497e0018-ccb8-45cf-b799-e98400f1ef6f', 'volume_name': 'volume-497e0018-ccb8-45cf-b799-e98400f1ef6f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:41.580513 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (583a60f6-ffa2-4f6f-85c1-2a47cbd81d36) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:41.597001 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (583a60f6-ffa2-4f6f-85c1-2a47cbd81d36) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:41.606742 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (732ffed2-f6cf-401b-8b96-649ece974ac7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:41.611215 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Volume 497e0018-ccb8-45cf-b799-e98400f1ef6f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-497e0018-ccb8-45cf-b799-e98400f1ef6f', 'volume_size': 1} May 28 21:07:41.616290 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] creating volume 'volume-497e0018-ccb8-45cf-b799-e98400f1ef6f' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:07:41.619172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:43.020542 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:07:43.021095 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:07:43.034702 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (732ffed2-f6cf-401b-8b96-649ece974ac7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-497e0018-ccb8-45cf-b799-e98400f1ef6f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:43.056148 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0f1bfd72-5f1f-4bde-ad58-01a9e305ab2b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:43.066513 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=21,cinder:INSERT=3,cinder:UPDATE=11 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:43.166264 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Volume volume-497e0018-ccb8-45cf-b799-e98400f1ef6f (497e0018-ccb8-45cf-b799-e98400f1ef6f): created successfully May 28 21:07:43.171531 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0f1bfd72-5f1f-4bde-ad58-01a9e305ab2b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:43.178454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Flow 'volume_create_manager' (5e3a7ea8-d502-409b-a1a0-4063bea8c699) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:43.260679 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0771543b-bc7e-4bb5-941d-2adf9ab40a88 tempest-VolumesAdminNegativeTest-1934144314 None] Created volume successfully. May 28 21:07:45.862112 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpysoab3df volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 45.116s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:07:46.045735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:46.277035 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-61ef832e-17b4-46d8-8edd-ec1ab8d5e787 req-d9409bfb-4422-4170-98c9-16ee085bcb2d tempest-VolumesAdminNegativeTest-1934144314 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:07:46.995668 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:07:46.996318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:07:46.996616 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Downloaded image 9586d0d3-c540-4636-bb36-06bb4b16c195 ((None, None)) to volume bdc702a5-c0fa-423b-b6bf-14c735b9af34 successfully. {{(pid=135921) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1221}} May 28 21:07:47.193469 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Temporary image 9586d0d3-c540-4636-bb36-06bb4b16c195 for user fc6ee6dba5d24e90bff48384539fcd7f is deleted. {{(pid=135921) fetch /opt/stack/cinder/cinder/image/image_utils.py:1092}} May 28 21:07:47.377526 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-61ef832e-17b4-46d8-8edd-ec1ab8d5e787 req-d9409bfb-4422-4170-98c9-16ee085bcb2d tempest-VolumesAdminNegativeTest-1934144314 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.100s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:07:47.382834 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-61ef832e-17b4-46d8-8edd-ec1ab8d5e787 req-d9409bfb-4422-4170-98c9-16ee085bcb2d tempest-VolumesAdminNegativeTest-1934144314 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-497e0018-ccb8-45cf-b799-e98400f1ef6f', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '497e0018-ccb8-45cf-b799-e98400f1ef6f', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:07:47.527201 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-61ef832e-17b4-46d8-8edd-ec1ab8d5e787 req-d9409bfb-4422-4170-98c9-16ee085bcb2d tempest-VolumesAdminNegativeTest-1934144314 None] Connection info returned from driver {'name': 'volumes/volume-497e0018-ccb8-45cf-b799-e98400f1ef6f', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '497e0018-ccb8-45cf-b799-e98400f1ef6f', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3947e77e-5904-47d3-9c97-e6c706d1fbf5'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:07:47.708833 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Created reservations ['14947bbf-28bf-49ad-b372-d4f1e5e21835', '4a69e6d6-de52-40cb-afcc-2ea470304c65', '39d282c6-0c4f-4b36-bb73-464c70211e92', 'd7010443-8779-458f-b6d7-70872f6cfdf1'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:07:47.735063 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Creating image volume entry: {'metadata': {}, 'multiattach': False, 'updated_at': datetime.datetime(2022, 5, 28, 21, 6, 25, tzinfo=datetime.timezone.utc), 'group_id': None, 'replication_driver_data': None, 'consistencygroup_id': None, 'bootable': False, 'host': 'ncdv-master-1@ceph#ceph', 'cluster_name': None, 'provider_auth': None, 'shared_targets': True, 'terminated_at': None, 'display_description': None, 'volume_glance_metadata': [{'key': 'signature_verified', 'value': 'False'}], 'volume_admin_metadata': [], 'availability_zone': 'nova', 'previous_status': None, 'replication_extended_status': None, 'scheduled_at': datetime.datetime(2022, 5, 28, 21, 6, 25, tzinfo=datetime.timezone.utc), 'provider_id': None, 'user_id': 'fc6ee6dba5d24e90bff48384539fcd7f', 'created_at': datetime.datetime(2022, 5, 28, 21, 6, 24, tzinfo=datetime.timezone.utc), 'launched_at': None, 'service_uuid': None, 'provider_geometry': None, 'encryption_key_id': None, 'replication_status': None, 'ec2_id': None, 'size': 1, 'provider_location': None, 'deleted': False, 'display_name': 'image-9586d0d3-c540-4636-bb36-06bb4b16c195', 'project_id': '3186f9d3c2b341afa8b098ebcaaf974b', 'volume_metadata': [], 'deleted_at': None, 'volume_type_id': 'baa4f94a-73f9-4a6a-905a-4a2e4bce80ff', 'attach_status': 'detached', 'status': 'creating', 'source_volid': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34'}. {{(pid=135921) _clone_image_volume /opt/stack/cinder/cinder/volume/manager.py:1620}} May 28 21:07:47.805149 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-61ef832e-17b4-46d8-8edd-ec1ab8d5e787 req-d9409bfb-4422-4170-98c9-16ee085bcb2d tempest-VolumesAdminNegativeTest-1934144314 None] attachment_update completed successfully. May 28 21:07:48.113419 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume reschedule parameters: False retry: None {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:07:48.148683 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Flow 'volume_create_manager' (5c488fcf-dc7a-49dd-ba4a-2f9353e1795a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:48.160012 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5505491f-60ad-4c42-bbc9-40dddbf0f70f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:48.237566 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5505491f-60ad-4c42-bbc9-40dddbf0f70f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:06:24Z,deleted=False,deleted_at=None,display_description=None,display_name='image-9586d0d3-c540-4636-bb36-06bb4b16c195',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=d155253c-0f01-4ae8-9e2c-11ffe940469e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='3186f9d3c2b341afa8b098ebcaaf974b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:06:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=bdc702a5-c0fa-423b-b6bf-14c735b9af34,status='creating',terminated_at=None,updated_at=2022-05-28T21:06:25Z,use_quota=True,user_id='fc6ee6dba5d24e90bff48384539fcd7f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:48.252833 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (804a12b9-1a0c-4e14-a089-c2d29394bb56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:48.262686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (804a12b9-1a0c-4e14-a089-c2d29394bb56) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:48.272646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (34afbd25-1238-4b64-90ea-2b16d5f0d504) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:48.306120 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (34afbd25-1238-4b64-90ea-2b16d5f0d504) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'volume_name': 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e', 'volume_size': 1, 'source_volid': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'source_volstatus': 'downloading'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:48.320406 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c75758d6-7f59-481e-9124-3e3b1c18afc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:48.330657 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c75758d6-7f59-481e-9124-3e3b1c18afc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:48.338185 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff7513fc-ebbc-4776-91fb-4a7a83bade0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:48.339325 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume d155253c-0f01-4ae8-9e2c-11ffe940469e: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e', 'volume_size': 1, 'source_volid': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'source_volstatus': 'downloading'} May 28 21:07:48.376999 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:48.583430 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] creating snapshot='volume-d155253c-0f01-4ae8-9e2c-11ffe940469e.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:07:50.344517 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] cloning 'volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34@volume-d155253c-0f01-4ae8-9e2c-11ffe940469e.clone_snap' to 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:07:51.469199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:07:51.471160 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:07:51.471789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:07:51.781348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:07:51.785249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ff7513fc-ebbc-4776-91fb-4a7a83bade0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e', 'volume_size': 1, 'source_volid': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'source_volstatus': 'downloading'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:51.789750 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f6739a61-f36b-4cc5-815d-1139b0f49401) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:51.868654 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-d155253c-0f01-4ae8-9e2c-11ffe940469e (d155253c-0f01-4ae8-9e2c-11ffe940469e): created successfully May 28 21:07:51.882326 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f6739a61-f36b-4cc5-815d-1139b0f49401) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:51.893896 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Flow 'volume_create_manager' (5c488fcf-dc7a-49dd-ba4a-2f9353e1795a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:51.982881 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Created volume successfully. May 28 21:07:52.177395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Creating new image-volume cache entry for image 9586d0d3-c540-4636-bb36-06bb4b16c195 on ncdv-master-1@ceph#ceph {{(pid=135921) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:104}} May 28 21:07:52.258541 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] New image-volume cache entry created: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 7, 52, 181152)}. {{(pid=135921) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:129}} May 28 21:07:52.265186 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 86.823s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:07:52.266655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Marking volume bdc702a5-c0fa-423b-b6bf-14c735b9af34 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:07:52.364682 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to bdc702a5-c0fa-423b-b6bf-14c735b9af34. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:07:52.365479 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Creating volume glance metadata for volume bdc702a5-c0fa-423b-b6bf-14c735b9af34 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:07:52.402053 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 44.579s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:07:52.429629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 7, 52, 423044)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:07:52.430333 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:07:52.431035 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:07:52.469508 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.068s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:07:52.474908 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:07:52.501481 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 7, 52, 488963)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:07:52.504159 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:07:52.504732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:07:52.539440 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:52.760104 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] creating snapshot='volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:07:54.515033 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e.clone_snap' to 'volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:07:54.569886 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-afa4fe71-c57f-4332-9ccc-a90f47cdb774 req-ff8d7732-d36f-4ea4-8d1d-ca92d1faa410 tempest-VolumesAdminNegativeTest-1934144314 None] Terminate volume connection completed successfully. May 28 21:07:54.570430 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-afa4fe71-c57f-4332-9ccc-a90f47cdb774 req-ff8d7732-d36f-4ea4-8d1d-ca92d1faa410 tempest-VolumesAdminNegativeTest-1934144314 None] Deleting attachment 3947e77e-5904-47d3-9c97-e6c706d1fbf5. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:07:54.798989 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b451a8ce-67e0-4d11-8439-d2d6459195d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:54.804097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7490988a-7026-4a73-8f96-8c92db1d6949) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:54.902975 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 (bdc702a5-c0fa-423b-b6bf-14c735b9af34): created successfully May 28 21:07:54.906600 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7490988a-7026-4a73-8f96-8c92db1d6949) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:54.923862 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Flow 'volume_create_manager' (ede803e5-67a6-4df9-82fb-b5089984e324) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:55.002127 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6bca207b-1707-41b3-9b4c-da1101b6a41b tempest-LiveAutoBlockMigrationV225Test-605147657 None] Created volume successfully. May 28 21:07:55.860925 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:07:55.862244 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:07:55.862622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:07:56.759542 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:07:56.760870 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Marking volume 69bfe0ab-8925-40f5-bc39-33e09bf78a5e as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:07:56.858199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 69bfe0ab-8925-40f5-bc39-33e09bf78a5e. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:07:56.858642 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Creating volume glance metadata for volume 69bfe0ab-8925-40f5-bc39-33e09bf78a5e backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:07:56.923145 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=67,cinder:UPDATE=14,cinder:DELETE=2,cinder:INSERT=17 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:57.058573 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6dd4b542-e323-4966-8d6d-bd348b44554a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:57.063913 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3c18fcca-45a6-4655-8234-5b014277d167) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:57.196569 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Volume volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e (69bfe0ab-8925-40f5-bc39-33e09bf78a5e): created successfully May 28 21:07:57.206218 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3c18fcca-45a6-4655-8234-5b014277d167) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:57.221820 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Flow 'volume_create_manager' (6196fd31-ca4a-4b0e-8fac-ad3a3ed8171b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:57.291377 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-7bee85ed-a0b8-4a1e-9ac0-c9bf5cc07222 tempest-LiveMigrationTest-1347342374 None] Created volume successfully. May 28 21:07:57.929590 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=62,cinder:UPDATE=17,cinder:INSERT=19,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:07:58.012343 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:07:58.066353 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Flow 'volume_create_manager' (eedd9d17-2616-4090-9741-783b32f93eb4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:58.077759 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (759108b9-73ed-46a3-9289-86f7748c6ef5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:58.106972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (759108b9-73ed-46a3-9289-86f7748c6ef5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:07:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-2079426095',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e112d418-9504-4ab3-93ed-83997b63a15b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f7a9c4408ec1432cb05c5256657886d4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:07:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:07:58Z,use_quota=True,user_id='be94d10c650645658c755b7fc1595370',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:58.119211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6699c941-64b0-44d0-8d53-9ce7d61e9653) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:58.126531 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6699c941-64b0-44d0-8d53-9ce7d61e9653) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:58.134482 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3435e3ce-9617-4619-9a77-32b8b9ede1d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:58.146551 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3435e3ce-9617-4619-9a77-32b8b9ede1d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e112d418-9504-4ab3-93ed-83997b63a15b', 'volume_name': 'volume-e112d418-9504-4ab3-93ed-83997b63a15b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:58.153690 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fdfc8aa2-a382-4e41-bcbb-248f89248bda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:58.163645 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fdfc8aa2-a382-4e41-bcbb-248f89248bda) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:58.169110 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c961eaac-497b-48a1-87bd-f4bc2a53d65a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:58.170794 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Volume e112d418-9504-4ab3-93ed-83997b63a15b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e112d418-9504-4ab3-93ed-83997b63a15b', 'volume_size': 1} May 28 21:07:58.171572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] creating volume 'volume-e112d418-9504-4ab3-93ed-83997b63a15b' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:07:58.172558 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:07:58.826702 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:07:58.827211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:07:58.831780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c961eaac-497b-48a1-87bd-f4bc2a53d65a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e112d418-9504-4ab3-93ed-83997b63a15b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:58.839506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (567bdfe3-af31-4d5a-94c6-c09afb9ccd3e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:07:58.942545 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Volume volume-e112d418-9504-4ab3-93ed-83997b63a15b (e112d418-9504-4ab3-93ed-83997b63a15b): created successfully May 28 21:07:58.947166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (567bdfe3-af31-4d5a-94c6-c09afb9ccd3e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:07:58.957712 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Flow 'volume_create_manager' (eedd9d17-2616-4090-9741-783b32f93eb4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:07:59.066950 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0f10c733-4d03-4e7a-bc07-de06eeb1d657 tempest-VolumesAdminNegativeTest-1934144314 None] Created volume successfully. May 28 21:08:00.206402 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] Lock "b'cinder-e112d418-9504-4ab3-93ed-83997b63a15b-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:08:00.373065 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:00.621940 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:08:00.622487 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] Volume volume-e112d418-9504-4ab3-93ed-83997b63a15b is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:08:00.777914 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] deleting rbd volume volume-e112d418-9504-4ab3-93ed-83997b63a15b {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:08:00.784349 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:08:00.787349 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:00.924404 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:00.964729 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:08:00.986815 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-685dc7ea-2081-43a0-811f-fd8cf0174aa3 req-67ec1ffc-38ad-49b8-9268-fea576debd93 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:08:01.787341 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-685dc7ea-2081-43a0-811f-fd8cf0174aa3 req-67ec1ffc-38ad-49b8-9268-fea576debd93 tempest-LiveAutoBlockMigrationV225Test-605147657 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.800s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:08:01.788916 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-685dc7ea-2081-43a0-811f-fd8cf0174aa3 req-67ec1ffc-38ad-49b8-9268-fea576debd93 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:08:01.885491 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-685dc7ea-2081-43a0-811f-fd8cf0174aa3 req-67ec1ffc-38ad-49b8-9268-fea576debd93 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Connection info returned from driver {'name': 'volumes/volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3d10de6f-50c1-4791-9ee9-67824a4889ee'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:08:02.099207 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-685dc7ea-2081-43a0-811f-fd8cf0174aa3 req-67ec1ffc-38ad-49b8-9268-fea576debd93 tempest-LiveAutoBlockMigrationV225Test-605147657 None] attachment_update completed successfully. May 28 21:08:03.212416 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c815e37f-7eb2-420a-8c22-c2e4603c955f req-a2123117-38e3-4991-9f15-127c0c0266d0 tempest-LiveMigrationTest-1347342374 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:08:04.287529 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c815e37f-7eb2-420a-8c22-c2e4603c955f req-a2123117-38e3-4991-9f15-127c0c0266d0 tempest-LiveMigrationTest-1347342374 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.075s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:08:04.288345 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c815e37f-7eb2-420a-8c22-c2e4603c955f req-a2123117-38e3-4991-9f15-127c0c0266d0 tempest-LiveMigrationTest-1347342374 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:08:04.471894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c815e37f-7eb2-420a-8c22-c2e4603c955f req-a2123117-38e3-4991-9f15-127c0c0266d0 tempest-LiveMigrationTest-1347342374 None] Connection info returned from driver {'name': 'volumes/volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '17922ff3-3408-4a09-a5fd-edeabb1695ac'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:08:04.761748 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c815e37f-7eb2-420a-8c22-c2e4603c955f req-a2123117-38e3-4991-9f15-127c0c0266d0 tempest-LiveMigrationTest-1347342374 None] attachment_update completed successfully. May 28 21:08:04.848781 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] Created reservations ['1ecba96c-4aa3-4ece-8051-39b38b807983', 'afacda16-0ac3-4a3d-8aa4-6c7ca738b69d', 'c6c2434c-514a-41d8-9f1f-ea2979970bb7', 'f2a83a0e-7241-43d7-9334-772322e9b314'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:08:05.143837 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:05.233324 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:05.307685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:08:05.313710 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] Deleted volume successfully. May 28 21:08:05.425530 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-fe09048e-74ce-4a58-8241-4bdff9e44193 tempest-VolumesAdminNegativeTest-1934144314 None] Lock "b'cinder-e112d418-9504-4ab3-93ed-83997b63a15b-delete_volume'" released by "delete_volume" :: held 5.219s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:08:06.024099 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] Lock "b'cinder-497e0018-ccb8-45cf-b799-e98400f1ef6f-delete_volume'" acquired by "delete_volume" :: waited 0.022s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:08:06.213799 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:06.393790 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:08:06.394385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] Volume volume-497e0018-ccb8-45cf-b799-e98400f1ef6f is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:08:06.486141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] deleting rbd volume volume-497e0018-ccb8-45cf-b799-e98400f1ef6f {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:08:10.102841 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] Created reservations ['1c1e245b-7bbc-4650-8ef1-f621cdb879e0', '259b7442-ff3f-4d30-b8e2-c94b6ef26b05', '30aed4dd-e5fb-4729-a5bc-b74c221b1c0a', 'd7e63fa6-d892-49c4-addb-0f0c2c974c3f'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:08:10.268838 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:10.304074 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:10.331269 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:08:10.339543 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] Deleted volume successfully. May 28 21:08:10.398331 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5b741fd4-96b3-4573-b3af-fce1c1b2ba21 tempest-VolumesAdminNegativeTest-1934144314 None] Lock "b'cinder-497e0018-ccb8-45cf-b799-e98400f1ef6f-delete_volume'" released by "delete_volume" :: held 4.385s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:08:20.345818 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=64,cinder:UPDATE=23,cinder:DELETE=2,cinder:INSERT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:31.725212 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-95b4887e-309a-4d7e-b0f9-663fe29370ef tempest-LiveAutoBlockMigrationV225Test-664987053 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:08:32.497332 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-95b4887e-309a-4d7e-b0f9-663fe29370ef tempest-LiveAutoBlockMigrationV225Test-664987053 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.772s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:08:32.498302 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-95b4887e-309a-4d7e-b0f9-663fe29370ef tempest-LiveAutoBlockMigrationV225Test-664987053 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:08:32.514726 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=49,cinder:UPDATE=21,cinder:INSERT=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:08:32.611195 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-95b4887e-309a-4d7e-b0f9-663fe29370ef tempest-LiveAutoBlockMigrationV225Test-664987053 None] Connection info returned from driver {'name': 'volumes/volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'bdc702a5-c0fa-423b-b6bf-14c735b9af34', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '03807888-a5d1-46e3-993a-d2b2bf8db87b'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:08:32.751344 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-95b4887e-309a-4d7e-b0f9-663fe29370ef tempest-LiveAutoBlockMigrationV225Test-664987053 None] attachment_update completed successfully. May 28 21:08:43.943321 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-a43f8d62-ec11-4e9a-b83b-f5363f82fe36 tempest-LiveAutoBlockMigrationV225Test-664987053 None] Terminate volume connection completed successfully. May 28 21:08:43.944347 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-41817c2a-e50d-48e8-99f8-a64e6506cb6c req-a43f8d62-ec11-4e9a-b83b-f5363f82fe36 tempest-LiveAutoBlockMigrationV225Test-664987053 None] Deleting attachment 3d10de6f-50c1-4791-9ee9-67824a4889ee. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:08:53.017776 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-132ddf73-986d-479e-9eb8-49bb1f9401c0 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Terminate volume connection completed successfully. May 28 21:08:53.019078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-132ddf73-986d-479e-9eb8-49bb1f9401c0 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Deleting attachment 03807888-a5d1-46e3-993a-d2b2bf8db87b. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:08:54.539053 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Lock "b'cinder-bdc702a5-c0fa-423b-b6bf-14c735b9af34-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:08:54.636280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:54.792932 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:08:54.793293 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] volume has clone snapshot(s) {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1294}} May 28 21:08:54.794642 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:08:55.136097 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-55a27350-416e-4740-9283-983437eb2c0d req-42ea3c00-ae94-4668-8a8f-180772f4e8d5 tempest-LiveMigrationTest-1002282877 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:08:56.169130 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-55a27350-416e-4740-9283-983437eb2c0d req-42ea3c00-ae94-4668-8a8f-180772f4e8d5 tempest-LiveMigrationTest-1002282877 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.033s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:08:56.170088 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-55a27350-416e-4740-9283-983437eb2c0d req-42ea3c00-ae94-4668-8a8f-180772f4e8d5 tempest-LiveMigrationTest-1002282877 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:08:56.279172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-55a27350-416e-4740-9283-983437eb2c0d req-42ea3c00-ae94-4668-8a8f-180772f4e8d5 tempest-LiveMigrationTest-1002282877 None] Connection info returned from driver {'name': 'volumes/volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '69bfe0ab-8925-40f5-bc39-33e09bf78a5e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0c4de084-f01a-47c2-a0f3-9f66ff6d3ba0'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:08:56.349594 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Lock "b'cinder-96682203-c2c9-4aeb-9901-07faacbd380d-delete_volume'" acquired by "delete_volume" :: waited 0.016s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:08:56.429709 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-55a27350-416e-4740-9283-983437eb2c0d req-42ea3c00-ae94-4668-8a8f-180772f4e8d5 tempest-LiveMigrationTest-1002282877 None] attachment_update completed successfully. May 28 21:08:56.521973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:56.741659 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:08:56.742144 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Volume volume-96682203-c2c9-4aeb-9901-07faacbd380d is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:08:56.978164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] deleting rbd volume volume-96682203-c2c9-4aeb-9901-07faacbd380d {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:08:57.579483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Created reservations ['549207dc-7319-4de9-9476-2876ce5a82c3', '71b91c44-8e21-4f2b-966d-0dbda816e2d9', '319b5e68-9836-43fd-aa65-f12b5c7af553', 'dd0771dc-0c38-426a-b3ec-b2817f7ced60'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:08:57.827496 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:57.851521 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:08:57.874248 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:08:57.879007 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Deleted volume successfully. May 28 21:08:57.973407 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-29f7c8e4-6b2f-46e9-a867-57138fe66a3b req-bbbe1ff1-7e8a-485a-89df-299c90a60772 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Lock "b'cinder-bdc702a5-c0fa-423b-b6bf-14c735b9af34-delete_volume'" released by "delete_volume" :: held 3.435s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:09:00.782135 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:09:00.782856 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:00.861164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:00.929573 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:09:01.150900 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=66,cinder:UPDATE=14,cinder:INSERT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:05.530942 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=30,cinder:UPDATE=13,cinder:INSERT=1,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:05.655756 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Created reservations ['1d7b5b47-ca1f-46f0-b890-02f6aed43a07', '9e94c09f-b5d4-4a49-a6f7-238870c429b1', '8cc190ff-1253-469c-9cc8-8116f1ccb997', '5b6182d2-fd80-4182-bd4c-2514ede78516'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:09:05.902032 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:05.965738 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:06.073764 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:09:06.083671 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Deleted volume successfully. May 28 21:09:06.245134 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ab3a8428-36e2-40f4-a562-c1b9d4639be3 tempest-LiveAutoBlockMigrationV225Test-605147657 None] Lock "b'cinder-96682203-c2c9-4aeb-9901-07faacbd380d-delete_volume'" released by "delete_volume" :: held 9.896s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:09:06.805498 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-55a27350-416e-4740-9283-983437eb2c0d req-f350c244-ad48-4417-8af5-973e8d0f4850 tempest-LiveMigrationTest-1002282877 None] Terminate volume connection completed successfully. May 28 21:09:06.812876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-55a27350-416e-4740-9283-983437eb2c0d req-f350c244-ad48-4417-8af5-973e8d0f4850 tempest-LiveMigrationTest-1002282877 None] Deleting attachment 17922ff3-3408-4a09-a5fd-edeabb1695ac. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:09:18.326966 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-e9c28869-86c3-47be-83ca-723052db7daa tempest-LiveMigrationTest-1347342374 None] Terminate volume connection completed successfully. May 28 21:09:18.327596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-e9c28869-86c3-47be-83ca-723052db7daa tempest-LiveMigrationTest-1347342374 None] Deleting attachment 0c4de084-f01a-47c2-a0f3-9f66ff6d3ba0. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:09:18.943686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] Lock "b'cinder-69bfe0ab-8925-40f5-bc39-33e09bf78a5e-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:09:19.103528 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:19.365108 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:09:19.536628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] deleting rbd volume volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:09:22.056615 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] Lock "b'cinder-b107b45a-f2d1-4795-8559-3836162f6660-delete_volume'" acquired by "delete_volume" :: waited 0.037s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:09:22.207934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:22.655313 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:09:22.656041 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] Volume volume-b107b45a-f2d1-4795-8559-3836162f6660 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:09:22.850926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] deleting rbd volume volume-b107b45a-f2d1-4795-8559-3836162f6660 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:09:24.755151 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:09:24.960233 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] deleting parent snapshot volume-69bfe0ab-8925-40f5-bc39-33e09bf78a5e.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:09:28.314366 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] Created reservations ['d5135ec0-e551-4fdd-bd74-6f6fc5bb92f9', 'd1445794-ed1f-4fa7-8d85-db5ed46262d2', 'cb942095-f5b6-4b3f-aa80-b8bbdbba2a68', '76610e20-5ead-4f24-a1a3-59115a183313'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:09:28.707637 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:28.815139 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:28.935351 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:09:28.954526 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] Deleted volume successfully. May 28 21:09:29.045723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-c906982d-fadf-4d4a-b5a3-f325af7b2bb9 req-7383d539-8d59-47d9-b327-93ff343a14a6 tempest-LiveMigrationTest-1347342374 None] Lock "b'cinder-69bfe0ab-8925-40f5-bc39-33e09bf78a5e-delete_volume'" released by "delete_volume" :: held 10.102s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:09:29.283733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] Created reservations ['12192678-1cd7-4da1-8b93-ae5e937852cb', '426b4b04-588f-4d88-b27b-8d9e2e878f08', '60af2518-60af-4ffa-8fb8-8d1dfe371938', '69945bb9-998c-4f05-ade1-00cccf4fec8a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:09:29.551306 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:29.630450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:09:29.705265 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:09:29.709632 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] Deleted volume successfully. May 28 21:09:29.807735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a5d2cbc1-407c-4548-acc5-6e7b44d9908b tempest-LiveMigrationTest-1347342374 None] Lock "b'cinder-b107b45a-f2d1-4795-8559-3836162f6660-delete_volume'" released by "delete_volume" :: held 7.752s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:09:41.153721 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=49,cinder:INSERT=10,cinder:UPDATE=33,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:09:51.158132 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=37,cinder:UPDATE=18,cinder:INSERT=2,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:00.786334 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:10:00.787924 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:10:00.991316 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:10:01.132877 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:10:11.156611 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:21.151480 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:41.164090 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:47.601926 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:10:47.673482 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:10:47.760728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Flow 'volume_create_manager' (3010405f-683f-40e0-bba6-b8805a839a4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:10:47.783906 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ca5cfab-c076-48db-b6b8-357b6efe94e3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:10:47.815967 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0ca5cfab-c076-48db-b6b8-357b6efe94e3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:10:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-2045960363',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=305738dc-96e3-44a1-968b-a25b737a3d47,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b4be72b7f8c54528b1c0a2a1505416c1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:10:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:10:47Z,use_quota=True,user_id='94f224e3fd2142d0991cb173e2a2c880',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:10:47.823923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7adc6235-8a43-483f-aaae-bfec1a76439d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:10:47.832635 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7adc6235-8a43-483f-aaae-bfec1a76439d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:10:47.841211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3943786a-242b-4722-9b00-fe2f69ccb076) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:10:48.001623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3943786a-242b-4722-9b00-fe2f69ccb076) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '305738dc-96e3-44a1-968b-a25b737a3d47', 'volume_name': 'volume-305738dc-96e3-44a1-968b-a25b737a3d47', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:10:48.007416 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5eb25a8c-2015-435a-9c98-70a5a688a793) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:10:48.033128 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5eb25a8c-2015-435a-9c98-70a5a688a793) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:10:48.042089 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cf54002c-0233-4371-9094-bb71640ab18b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:10:48.044737 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Volume 305738dc-96e3-44a1-968b-a25b737a3d47: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-305738dc-96e3-44a1-968b-a25b737a3d47', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:10:48.044737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Cloning 305738dc-96e3-44a1-968b-a25b737a3d47 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:10:48.084439 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.031s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:10:48.184681 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 10, 48, 94711)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:10:48.185130 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:10:48.185540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:10:48.192002 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.108s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:10:48.193280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:10:48.203547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 10, 48, 199513)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:10:48.204521 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:10:48.205079 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:10:48.239675 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:10:48.459215 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] creating snapshot='volume-305738dc-96e3-44a1-968b-a25b737a3d47.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:10:49.740272 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-305738dc-96e3-44a1-968b-a25b737a3d47.clone_snap' to 'volume-305738dc-96e3-44a1-968b-a25b737a3d47' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:10:51.332300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:10:51.332300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:10:51.332300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:10:51.895140 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:10:51.896003 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Marking volume 305738dc-96e3-44a1-968b-a25b737a3d47 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:10:52.007479 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 305738dc-96e3-44a1-968b-a25b737a3d47. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:10:52.008839 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Creating volume glance metadata for volume 305738dc-96e3-44a1-968b-a25b737a3d47 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:10:52.181355 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cf54002c-0233-4371-9094-bb71640ab18b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-305738dc-96e3-44a1-968b-a25b737a3d47', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:10:52.204917 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f05f060a-6460-4738-9172-6b43a67f034d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:10:52.327479 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Volume volume-305738dc-96e3-44a1-968b-a25b737a3d47 (305738dc-96e3-44a1-968b-a25b737a3d47): created successfully May 28 21:10:52.332242 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f05f060a-6460-4738-9172-6b43a67f034d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:10:52.340072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Flow 'volume_create_manager' (3010405f-683f-40e0-bba6-b8805a839a4f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:10:52.408151 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c0314e02-05df-45d8-9c17-083e1c221b04 tempest-ServersTestBootFromVolume-78716756 None] Created volume successfully. May 28 21:11:00.295056 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-71dad75e-b069-4965-afb2-392395a0540f req-7ed64977-2eaf-4aab-bf74-76d0597700fa tempest-ServersTestBootFromVolume-78716756 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:11:00.790131 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:11:00.791555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:11:00.882258 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:11:00.956734 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:11:01.271140 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-71dad75e-b069-4965-afb2-392395a0540f req-7ed64977-2eaf-4aab-bf74-76d0597700fa tempest-ServersTestBootFromVolume-78716756 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.976s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:11:01.274203 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-71dad75e-b069-4965-afb2-392395a0540f req-7ed64977-2eaf-4aab-bf74-76d0597700fa tempest-ServersTestBootFromVolume-78716756 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-305738dc-96e3-44a1-968b-a25b737a3d47', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '305738dc-96e3-44a1-968b-a25b737a3d47', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:11:01.418508 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-71dad75e-b069-4965-afb2-392395a0540f req-7ed64977-2eaf-4aab-bf74-76d0597700fa tempest-ServersTestBootFromVolume-78716756 None] Connection info returned from driver {'name': 'volumes/volume-305738dc-96e3-44a1-968b-a25b737a3d47', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '305738dc-96e3-44a1-968b-a25b737a3d47', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'eaa9fe51-c32a-4ac0-8e73-abc037744cc6'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:11:01.590304 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-71dad75e-b069-4965-afb2-392395a0540f req-7ed64977-2eaf-4aab-bf74-76d0597700fa tempest-ServersTestBootFromVolume-78716756 None] attachment_update completed successfully. May 28 21:11:11.581432 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=30,cinder:UPDATE=7,cinder:INSERT=9 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:21.176344 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=30,cinder:UPDATE=5,cinder:INSERT=3,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:31.188239 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:11:39.145407 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:11:39.201361 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Flow 'volume_create_manager' (eca2afc3-7c90-4399-9e40-3383435ace13) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:11:39.208123 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f97fd570-246c-4d62-ac44-240c5f4388bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:11:39.249463 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f97fd570-246c-4d62-ac44-240c5f4388bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:11:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-DeleteServersTestJSON-volume-1096881903',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=97e7aab3-d275-463e-aaa5-5830a8126a8d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='cf04fd760c54473f83980a385252fa3c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:11:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:11:39Z,use_quota=True,user_id='bf9c8f66e7d24ca9baca835e41546f22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:11:39.257815 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7e25f57e-6010-4e09-9ac3-06935c099475) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:11:39.265528 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7e25f57e-6010-4e09-9ac3-06935c099475) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:11:39.287869 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f3d5c997-7a8d-426e-a03c-70832402aaec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:11:39.294077 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f3d5c997-7a8d-426e-a03c-70832402aaec) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '97e7aab3-d275-463e-aaa5-5830a8126a8d', 'volume_name': 'volume-97e7aab3-d275-463e-aaa5-5830a8126a8d', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:11:39.300193 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d569c8ce-d066-42d3-83d0-ffc87e6acbce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:11:39.305149 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d569c8ce-d066-42d3-83d0-ffc87e6acbce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:11:39.310692 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8b75ed9-4ddf-459f-b467-cce4f5754fad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:11:39.312287 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Volume 97e7aab3-d275-463e-aaa5-5830a8126a8d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-97e7aab3-d275-463e-aaa5-5830a8126a8d', 'volume_size': 1} May 28 21:11:39.312696 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] creating volume 'volume-97e7aab3-d275-463e-aaa5-5830a8126a8d' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:11:39.318231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:11:40.295897 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:11:40.296388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:11:40.300045 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8b75ed9-4ddf-459f-b467-cce4f5754fad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-97e7aab3-d275-463e-aaa5-5830a8126a8d', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:11:40.309077 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b4d1c09-1a75-4730-93c9-5fa776d326a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:11:40.394918 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Volume volume-97e7aab3-d275-463e-aaa5-5830a8126a8d (97e7aab3-d275-463e-aaa5-5830a8126a8d): created successfully May 28 21:11:40.398848 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6b4d1c09-1a75-4730-93c9-5fa776d326a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:11:40.404722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Flow 'volume_create_manager' (eca2afc3-7c90-4399-9e40-3383435ace13) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:11:40.551719 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-411ac2ae-a01f-471d-becb-b9bcc01e6ff4 tempest-DeleteServersTestJSON-435436475 None] Created volume successfully. May 28 21:11:42.740246 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-89c8dfcc-3ce2-47eb-9191-6ceecf2f2143 req-bcc4e2b2-bc78-4950-b4ec-0d07cd6044c5 tempest-DeleteServersTestJSON-435436475 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:11:44.097200 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-89c8dfcc-3ce2-47eb-9191-6ceecf2f2143 req-bcc4e2b2-bc78-4950-b4ec-0d07cd6044c5 tempest-DeleteServersTestJSON-435436475 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.367s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:11:44.098134 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-89c8dfcc-3ce2-47eb-9191-6ceecf2f2143 req-bcc4e2b2-bc78-4950-b4ec-0d07cd6044c5 tempest-DeleteServersTestJSON-435436475 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-97e7aab3-d275-463e-aaa5-5830a8126a8d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '97e7aab3-d275-463e-aaa5-5830a8126a8d', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:11:44.217728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-89c8dfcc-3ce2-47eb-9191-6ceecf2f2143 req-bcc4e2b2-bc78-4950-b4ec-0d07cd6044c5 tempest-DeleteServersTestJSON-435436475 None] Connection info returned from driver {'name': 'volumes/volume-97e7aab3-d275-463e-aaa5-5830a8126a8d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '97e7aab3-d275-463e-aaa5-5830a8126a8d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '34e97f3f-4bd4-484f-91a7-a6d4828a5aae'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:11:44.393544 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-89c8dfcc-3ce2-47eb-9191-6ceecf2f2143 req-bcc4e2b2-bc78-4950-b4ec-0d07cd6044c5 tempest-DeleteServersTestJSON-435436475 None] attachment_update completed successfully. May 28 21:11:52.011961 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-41241d60-7f56-4d43-ab6e-10f7aff0524c req-082cbb37-81e3-4dd4-8ba9-dcfb039a2ca5 tempest-DeleteServersTestJSON-435436475 None] Terminate volume connection completed successfully. May 28 21:11:52.017266 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-41241d60-7f56-4d43-ab6e-10f7aff0524c req-082cbb37-81e3-4dd4-8ba9-dcfb039a2ca5 tempest-DeleteServersTestJSON-435436475 None] Deleting attachment 34e97f3f-4bd4-484f-91a7-a6d4828a5aae. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:12:00.792804 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:12:00.794225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:12:00.860199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:12:00.987194 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:12:02.011401 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=19,cinder:UPDATE=6,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:05.442602 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=27,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:05.556468 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:12:05.581789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Flow 'volume_create_manager' (6362378a-c489-49f4-85dd-48a1015c871d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:12:05.601592 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8adb3479-e382-496e-8fcc-72abc854b2c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:05.633752 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8adb3479-e382-496e-8fcc-72abc854b2c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:12:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-2083439334',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f988fe9b-43f7-4336-8526-979ab3142238,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8f20390f83d3407e93a3d8d2a196281b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:12:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:12:05Z,use_quota=True,user_id='cafc81a185c645e893a77b7fd297d950',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:05.654387 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (82c7738c-91e0-492e-afc5-f780dce6007a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:05.660317 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (82c7738c-91e0-492e-afc5-f780dce6007a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:05.666578 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0686c251-5154-454d-b3bc-d65bfb8cce6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:05.833256 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0686c251-5154-454d-b3bc-d65bfb8cce6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'f988fe9b-43f7-4336-8526-979ab3142238', 'volume_name': 'volume-f988fe9b-43f7-4336-8526-979ab3142238', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:05.859657 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c0b711ce-d292-4f2f-a1e2-613df5ae64f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:05.877468 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c0b711ce-d292-4f2f-a1e2-613df5ae64f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:05.910603 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c0dee1e5-2142-46d5-bbbe-99e48743066b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:05.912753 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Volume f988fe9b-43f7-4336-8526-979ab3142238: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-f988fe9b-43f7-4336-8526-979ab3142238', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:12:05.914809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Cloning f988fe9b-43f7-4336-8526-979ab3142238 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:12:05.929737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:12:06.052554 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 12, 5, 934680)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:12:06.052921 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:12:06.053419 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:12:06.065669 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.131s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:12:06.065669 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:12:06.075385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 12, 6, 65097)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:12:06.075797 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:12:06.076187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:12:06.093141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:12:06.388332 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] creating snapshot='volume-f988fe9b-43f7-4336-8526-979ab3142238.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:12:08.754801 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-f988fe9b-43f7-4336-8526-979ab3142238.clone_snap' to 'volume-f988fe9b-43f7-4336-8526-979ab3142238' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:12:10.868168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:12:10.869311 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:12:10.869696 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:12:11.558633 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:12:11.559962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Marking volume f988fe9b-43f7-4336-8526-979ab3142238 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:12:11.704341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to f988fe9b-43f7-4336-8526-979ab3142238. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:12:11.708091 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Creating volume glance metadata for volume f988fe9b-43f7-4336-8526-979ab3142238 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:12:11.895661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c0dee1e5-2142-46d5-bbbe-99e48743066b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f988fe9b-43f7-4336-8526-979ab3142238', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:11.900931 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a3fcf674-db7f-42fe-82bb-f89aeb306759) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:11.990011 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Volume volume-f988fe9b-43f7-4336-8526-979ab3142238 (f988fe9b-43f7-4336-8526-979ab3142238): created successfully May 28 21:12:11.996870 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a3fcf674-db7f-42fe-82bb-f89aeb306759) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:12.005973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Flow 'volume_create_manager' (6362378a-c489-49f4-85dd-48a1015c871d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:12:12.096818 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2d6c5fbc-1533-437d-bd33-dc9365c40519 tempest-TaggedBootDevicesTest-427054523 None] Created volume successfully. May 28 21:12:13.773252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:12:13.793124 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Flow 'volume_create_manager' (08d784ce-f3ad-4e56-8d9d-3dc3078475f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:12:13.800299 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69a161ab-e6a5-42e5-86ed-ac733495e70e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:13.823570 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (69a161ab-e6a5-42e5-86ed-ac733495e70e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:12:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-353071513',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8f20390f83d3407e93a3d8d2a196281b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:12:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:12:14Z,use_quota=True,user_id='cafc81a185c645e893a77b7fd297d950',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:13.842617 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cdf8fd0d-1510-43d2-9e5c-c6589b1327a0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:13.857322 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cdf8fd0d-1510-43d2-9e5c-c6589b1327a0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:13.866501 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f5b51d0f-4100-4ad3-a0e5-fa9fff33d304) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:13.872948 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f5b51d0f-4100-4ad3-a0e5-fa9fff33d304) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'volume_name': 'volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:13.881064 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0e06b82-4bf7-4fc9-b7a1-09a9e43d82f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:13.887726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b0e06b82-4bf7-4fc9-b7a1-09a9e43d82f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:13.894952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c90159ce-e261-4377-bdbe-f5ffcd51f543) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:13.897245 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Volume 1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'volume_size': 1} May 28 21:12:13.898040 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] creating volume 'volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:12:13.899995 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:12:15.196690 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:12:15.197319 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:12:15.200852 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c90159ce-e261-4377-bdbe-f5ffcd51f543) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:15.205540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (efdccd37-2a0a-4a9b-949b-22a3566bdcc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:15.297008 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Volume volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615 (1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615): created successfully May 28 21:12:15.300922 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (efdccd37-2a0a-4a9b-949b-22a3566bdcc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:15.306722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Flow 'volume_create_manager' (08d784ce-f3ad-4e56-8d9d-3dc3078475f8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:12:15.400209 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fa72f0b7-3039-4f6b-861d-aea2c974fc38 tempest-TaggedBootDevicesTest-427054523 None] Created volume successfully. May 28 21:12:16.607855 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:12:16.628906 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Flow 'volume_create_manager' (7b17db86-ab9d-4b70-94e8-46dded67039e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:12:16.635633 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b3d21134-5316-43e8-bd7c-ba86ba716b3a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:16.651958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b3d21134-5316-43e8-bd7c-ba86ba716b3a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:12:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1052576711',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c7ec9972-c33d-41d5-bf85-a57642501bd3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8f20390f83d3407e93a3d8d2a196281b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:12:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:12:16Z,use_quota=True,user_id='cafc81a185c645e893a77b7fd297d950',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:16.657863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fa0f7d76-73cc-4c89-91fb-bea044d12192) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:16.661674 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (fa0f7d76-73cc-4c89-91fb-bea044d12192) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:16.667674 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1b801b89-8eba-4132-9555-961c08a7ee68) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:16.671782 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1b801b89-8eba-4132-9555-961c08a7ee68) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c7ec9972-c33d-41d5-bf85-a57642501bd3', 'volume_name': 'volume-c7ec9972-c33d-41d5-bf85-a57642501bd3', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:16.676393 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c096d406-7906-4ff8-a108-053bf2512ee0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:16.680339 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c096d406-7906-4ff8-a108-053bf2512ee0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:16.685179 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ad7f09f9-54b7-46f2-8796-e5700a32a330) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:16.686286 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Volume c7ec9972-c33d-41d5-bf85-a57642501bd3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c7ec9972-c33d-41d5-bf85-a57642501bd3', 'volume_size': 1} May 28 21:12:16.686664 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] creating volume 'volume-c7ec9972-c33d-41d5-bf85-a57642501bd3' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:12:16.687300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:12:17.700656 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:12:17.701288 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:12:17.705274 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ad7f09f9-54b7-46f2-8796-e5700a32a330) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c7ec9972-c33d-41d5-bf85-a57642501bd3', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:17.709813 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ceb69f65-dd19-429e-97d5-d9384e5597f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:12:17.784724 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Volume volume-c7ec9972-c33d-41d5-bf85-a57642501bd3 (c7ec9972-c33d-41d5-bf85-a57642501bd3): created successfully May 28 21:12:17.789989 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ceb69f65-dd19-429e-97d5-d9384e5597f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:12:17.798780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Flow 'volume_create_manager' (7b17db86-ab9d-4b70-94e8-46dded67039e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:12:17.915566 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-75967438-6ba4-4e77-861a-54929cb262e7 tempest-TaggedBootDevicesTest-427054523 None] Created volume successfully. May 28 21:12:31.575144 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=42,cinder:UPDATE=8,cinder:INSERT=12,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:41.580110 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=12,cinder:UPDATE=7,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:12:42.242407 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-598c1f9f-ec9b-415b-8963-055d3a0e8678 tempest-TaggedBootDevicesTest-427054523 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:12:43.121579 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-598c1f9f-ec9b-415b-8963-055d3a0e8678 tempest-TaggedBootDevicesTest-427054523 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.879s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:12:43.123748 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-598c1f9f-ec9b-415b-8963-055d3a0e8678 tempest-TaggedBootDevicesTest-427054523 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f988fe9b-43f7-4336-8526-979ab3142238', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'f988fe9b-43f7-4336-8526-979ab3142238', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:12:43.240447 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-598c1f9f-ec9b-415b-8963-055d3a0e8678 tempest-TaggedBootDevicesTest-427054523 None] Connection info returned from driver {'name': 'volumes/volume-f988fe9b-43f7-4336-8526-979ab3142238', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f988fe9b-43f7-4336-8526-979ab3142238', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '4c85200e-398a-4fcf-b339-c25a641cc399'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:12:43.415378 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-598c1f9f-ec9b-415b-8963-055d3a0e8678 tempest-TaggedBootDevicesTest-427054523 None] attachment_update completed successfully. May 28 21:12:45.128425 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-fa13645c-c510-4d31-b4ab-11010306492f tempest-TaggedBootDevicesTest-427054523 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:12:46.030389 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-fa13645c-c510-4d31-b4ab-11010306492f tempest-TaggedBootDevicesTest-427054523 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.901s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:12:46.031175 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-fa13645c-c510-4d31-b4ab-11010306492f tempest-TaggedBootDevicesTest-427054523 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:12:46.155430 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-fa13645c-c510-4d31-b4ab-11010306492f tempest-TaggedBootDevicesTest-427054523 None] Connection info returned from driver {'name': 'volumes/volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '494b2820-bd5e-44d5-aadd-27b830314cad'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:12:46.310525 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-fa13645c-c510-4d31-b4ab-11010306492f tempest-TaggedBootDevicesTest-427054523 None] attachment_update completed successfully. May 28 21:12:47.228044 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-df24581c-08f9-444a-9cbf-8ae63cdeeac3 tempest-TaggedBootDevicesTest-427054523 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:12:48.410376 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-df24581c-08f9-444a-9cbf-8ae63cdeeac3 tempest-TaggedBootDevicesTest-427054523 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.182s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:12:48.411388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-df24581c-08f9-444a-9cbf-8ae63cdeeac3 tempest-TaggedBootDevicesTest-427054523 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c7ec9972-c33d-41d5-bf85-a57642501bd3', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'c7ec9972-c33d-41d5-bf85-a57642501bd3', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:12:48.519226 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-df24581c-08f9-444a-9cbf-8ae63cdeeac3 tempest-TaggedBootDevicesTest-427054523 None] Connection info returned from driver {'name': 'volumes/volume-c7ec9972-c33d-41d5-bf85-a57642501bd3', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c7ec9972-c33d-41d5-bf85-a57642501bd3', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5c3711d3-fbc9-4589-8036-da112d6f1b65'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:12:48.693585 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c275b61d-09fd-417a-b0d2-2d115cd43248 req-df24581c-08f9-444a-9cbf-8ae63cdeeac3 tempest-TaggedBootDevicesTest-427054523 None] attachment_update completed successfully. May 28 21:12:58.689350 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=21,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:00.795593 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:13:00.796493 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:00.903006 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:00.999668 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:13:01.582155 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=48,cinder:UPDATE=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:21.560356 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:13:21.609029 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Flow 'volume_create_manager' (bf01c3da-18b9-49de-a856-121f14d5df20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:21.627212 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b82106a4-a06a-4d60-9aeb-2bd84dded67f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:21.699025 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b82106a4-a06a-4d60-9aeb-2bd84dded67f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:13:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-172169851',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=26cabed5-9ce1-42aa-b829-f68d323905f3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ca680361a7b349e39f77a7f7c96934f8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:13:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:13:21Z,use_quota=True,user_id='eb7c3a01b54f4db2841f4836635eff2e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:21.722356 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (933d997c-fd4c-4d09-8d06-ea2f7f6686f6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:21.726972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (933d997c-fd4c-4d09-8d06-ea2f7f6686f6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:21.740483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (be0eaf03-1768-4d71-a1c9-b5e5a80fa67b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:21.899106 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (be0eaf03-1768-4d71-a1c9-b5e5a80fa67b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '26cabed5-9ce1-42aa-b829-f68d323905f3', 'volume_name': 'volume-26cabed5-9ce1-42aa-b829-f68d323905f3', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:21.910719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (337bb672-5995-42b4-a736-2ff17ef87670) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:21.915596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (337bb672-5995-42b4-a736-2ff17ef87670) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:21.923026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0c6f549f-8a1c-4f60-b4aa-b30cb73e6946) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:21.931698 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume 26cabed5-9ce1-42aa-b829-f68d323905f3: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-26cabed5-9ce1-42aa-b829-f68d323905f3', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:13:21.931698 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Cloning 26cabed5-9ce1-42aa-b829-f68d323905f3 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:13:21.943752 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.018s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:13:22.003792 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 13, 21, 953090)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:13:22.004228 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:13:22.004832 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:13:22.010772 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.067s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:13:22.011652 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:13:22.040931 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 13, 22, 17956)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:13:22.040931 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:13:22.040931 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:13:22.083100 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:22.307000 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] creating snapshot='volume-26cabed5-9ce1-42aa-b829-f68d323905f3.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:13:24.233488 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-26cabed5-9ce1-42aa-b829-f68d323905f3.clone_snap' to 'volume-26cabed5-9ce1-42aa-b829-f68d323905f3' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:13:25.990587 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:13:25.994412 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:13:25.995083 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:13:26.657719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:13:26.657719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Marking volume 26cabed5-9ce1-42aa-b829-f68d323905f3 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:13:26.764530 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 26cabed5-9ce1-42aa-b829-f68d323905f3. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:13:26.765455 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Creating volume glance metadata for volume 26cabed5-9ce1-42aa-b829-f68d323905f3 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:13:27.010963 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0c6f549f-8a1c-4f60-b4aa-b30cb73e6946) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-26cabed5-9ce1-42aa-b829-f68d323905f3', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:27.036098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b06b71c7-3513-4b3d-a8c0-c0c36e769d77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:27.155413 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume volume-26cabed5-9ce1-42aa-b829-f68d323905f3 (26cabed5-9ce1-42aa-b829-f68d323905f3): created successfully May 28 21:13:27.159798 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b06b71c7-3513-4b3d-a8c0-c0c36e769d77) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:27.173648 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Flow 'volume_create_manager' (bf01c3da-18b9-49de-a856-121f14d5df20) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:27.270319 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f28a86f9-70a0-433d-b356-cba605e2cab4 tempest-TaggedBootDevicesTest_v242-1600206585 None] Created volume successfully. May 28 21:13:28.926188 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:13:28.946839 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Flow 'volume_create_manager' (9c18c4a3-835a-47c9-a2cb-873deacbfac9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:28.953600 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (380a4ba5-f23c-4682-a0f1-f878feb5afdc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:28.979888 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (380a4ba5-f23c-4682-a0f1-f878feb5afdc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:13:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-978442727',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=50730637-3c48-4270-bed2-229c25f4f463,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ca680361a7b349e39f77a7f7c96934f8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:13:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:13:29Z,use_quota=True,user_id='eb7c3a01b54f4db2841f4836635eff2e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:28.985273 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (02f4b1c9-3113-46c3-90e5-d110d2954b12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:28.989620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (02f4b1c9-3113-46c3-90e5-d110d2954b12) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:28.994777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f973b7d-e69a-49aa-8903-2fc021abf220) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:28.998774 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f973b7d-e69a-49aa-8903-2fc021abf220) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '50730637-3c48-4270-bed2-229c25f4f463', 'volume_name': 'volume-50730637-3c48-4270-bed2-229c25f4f463', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:29.004102 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5eb3e1bf-af0d-454e-b500-60c9b6a91968) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:29.007926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5eb3e1bf-af0d-454e-b500-60c9b6a91968) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:29.013062 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f3763906-8edd-4483-85fe-3646c628fb4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:29.014176 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume 50730637-3c48-4270-bed2-229c25f4f463: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-50730637-3c48-4270-bed2-229c25f4f463', 'volume_size': 1} May 28 21:13:29.014554 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] creating volume 'volume-50730637-3c48-4270-bed2-229c25f4f463' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:13:29.015216 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:29.951942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:13:29.953190 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:13:29.957330 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f3763906-8edd-4483-85fe-3646c628fb4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-50730637-3c48-4270-bed2-229c25f4f463', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:29.962951 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0526bc75-1f30-494e-b123-c82c94d8caf6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:30.035073 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume volume-50730637-3c48-4270-bed2-229c25f4f463 (50730637-3c48-4270-bed2-229c25f4f463): created successfully May 28 21:13:30.039335 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0526bc75-1f30-494e-b123-c82c94d8caf6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:30.045480 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Flow 'volume_create_manager' (9c18c4a3-835a-47c9-a2cb-873deacbfac9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:30.107072 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a7edcb55-96bd-41f0-b9e2-89b5052a38a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Created volume successfully. May 28 21:13:31.380451 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=27,cinder:UPDATE=7,cinder:INSERT=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:31.508108 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:13:31.532005 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Flow 'volume_create_manager' (420a665a-63c7-4dc4-a627-c730999bd3a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:31.538396 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a93890d8-5a2d-4da4-a0c8-1c9d9c29cfd0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.553912 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a93890d8-5a2d-4da4-a0c8-1c9d9c29cfd0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:13:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotsTestJSON-volume-981657982',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=290d5498-182e-4bc7-bcec-0cf0a2ec578a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='713293ceb9c44e8ea1a5d92668ad252a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:13:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:13:31Z,use_quota=True,user_id='041cd29a8fca40d0aab7e18c3b5e0dd6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.559695 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a58c7342-a2f9-4446-83da-68c44c07303c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.563676 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a58c7342-a2f9-4446-83da-68c44c07303c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.568612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b33ec34f-d45c-47bd-b1b5-62d71140df12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.573153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b33ec34f-d45c-47bd-b1b5-62d71140df12) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '290d5498-182e-4bc7-bcec-0cf0a2ec578a', 'volume_name': 'volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.577972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (001f1764-60ab-4835-93e7-762591eccb2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.581664 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (001f1764-60ab-4835-93e7-762591eccb2a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.586239 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (233f392c-0d85-4850-a20a-ef282d42d6cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.587898 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Volume 290d5498-182e-4bc7-bcec-0cf0a2ec578a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a', 'volume_size': 1} May 28 21:13:31.588286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] creating volume 'volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:13:31.588914 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:31.784510 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:13:31.803464 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Flow 'volume_create_manager' (0c880e96-815b-4144-b049-7831abe80016) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:31.810392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (964d3dc5-b97f-47b3-bc50-d4d8605ab617) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.826787 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (964d3dc5-b97f-47b3-bc50-d4d8605ab617) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:13:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1561598798',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0aae4e20-4ac4-4c87-a853-064fb70796fb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ca680361a7b349e39f77a7f7c96934f8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:13:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:13:32Z,use_quota=True,user_id='eb7c3a01b54f4db2841f4836635eff2e',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.832076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c57dd345-8e5c-47b7-ab3c-692b266fe6b5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.836421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c57dd345-8e5c-47b7-ab3c-692b266fe6b5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.841546 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1edf7c46-3843-4a83-8893-dac7a30536d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.847385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1edf7c46-3843-4a83-8893-dac7a30536d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0aae4e20-4ac4-4c87-a853-064fb70796fb', 'volume_name': 'volume-0aae4e20-4ac4-4c87-a853-064fb70796fb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.852222 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4758f3d4-e627-49c7-9e0f-612d5c94084e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.856953 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4758f3d4-e627-49c7-9e0f-612d5c94084e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:31.862977 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8d5e6b1d-f6b3-4f8f-8c38-67718e7436cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:31.864071 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume 0aae4e20-4ac4-4c87-a853-064fb70796fb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0aae4e20-4ac4-4c87-a853-064fb70796fb', 'volume_size': 1} May 28 21:13:31.864477 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] creating volume 'volume-0aae4e20-4ac4-4c87-a853-064fb70796fb' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:13:31.865129 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:32.281424 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:13:32.281825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:13:32.285531 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (233f392c-0d85-4850-a20a-ef282d42d6cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:32.292199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (55d7a985-c59d-4dee-a55b-91eff9885eef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:32.375654 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Volume volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a (290d5498-182e-4bc7-bcec-0cf0a2ec578a): created successfully May 28 21:13:32.380298 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (55d7a985-c59d-4dee-a55b-91eff9885eef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:32.386043 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Flow 'volume_create_manager' (420a665a-63c7-4dc4-a627-c730999bd3a9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:32.472071 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-ec12878e-6cf0-46b6-98c1-b1fe33d2c86e req-32c8d61c-5aae-4ae2-8474-6623cbd3c564 tempest-VolumesSnapshotsTestJSON-21813657 None] Created volume successfully. May 28 21:13:32.486308 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=35,cinder:UPDATE=9,cinder:INSERT=7,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:13:32.492760 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:13:32.493341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:13:32.497055 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8d5e6b1d-f6b3-4f8f-8c38-67718e7436cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0aae4e20-4ac4-4c87-a853-064fb70796fb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:32.501790 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (108ca67c-2de1-4f33-a4c7-07832151eb47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:13:32.678236 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume volume-0aae4e20-4ac4-4c87-a853-064fb70796fb (0aae4e20-4ac4-4c87-a853-064fb70796fb): created successfully May 28 21:13:32.682132 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (108ca67c-2de1-4f33-a4c7-07832151eb47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:13:32.687884 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Flow 'volume_create_manager' (0c880e96-815b-4144-b049-7831abe80016) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:13:32.780855 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8425847c-0e2c-4ea5-90c2-532dcaa2b95c tempest-TaggedBootDevicesTest_v242-1600206585 None] Created volume successfully. May 28 21:13:34.153556 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fe8b4d44-fe18-4fb5-b994-db98a9dd0906 req-0c080ce9-3c91-401a-80d0-4ca2e90f3900 tempest-VolumesSnapshotsTestJSON-21813657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:36.911960 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-fe8b4d44-fe18-4fb5-b994-db98a9dd0906 req-0c080ce9-3c91-401a-80d0-4ca2e90f3900 tempest-VolumesSnapshotsTestJSON-21813657 None] Create snapshot completed successfully May 28 21:13:38.626270 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-e02c0789-64b7-4bb9-9f03-f270c158eb9f req-2f5120d6-b503-469c-94de-48b958997f0e tempest-VolumesSnapshotsTestJSON-21813657 None] Lock "b'cinder-06b9663d-74c3-44b0-835e-37f8e32120ba-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.022s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:13:38.632543 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-e02c0789-64b7-4bb9-9f03-f270c158eb9f req-2f5120d6-b503-469c-94de-48b958997f0e tempest-VolumesSnapshotsTestJSON-21813657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:41.936639 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [req-e02c0789-64b7-4bb9-9f03-f270c158eb9f req-2f5120d6-b503-469c-94de-48b958997f0e tempest-VolumesSnapshotsTestJSON-21813657 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota. May 28 21:13:42.056100 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-e02c0789-64b7-4bb9-9f03-f270c158eb9f req-2f5120d6-b503-469c-94de-48b958997f0e tempest-VolumesSnapshotsTestJSON-21813657 None] Created reservations ['fd4d32ba-86b4-4514-955c-64e6c4aa1201', '37885bc7-40d5-48e2-9951-7f2fe5363776', 'f7491bbc-ca2a-4e4f-855c-4e79d2ff4e62', '9de55457-81e2-4511-b2cf-66f199075620'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:13:42.273025 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-e02c0789-64b7-4bb9-9f03-f270c158eb9f req-2f5120d6-b503-469c-94de-48b958997f0e tempest-VolumesSnapshotsTestJSON-21813657 None] Delete snapshot completed successfully. May 28 21:13:42.279662 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-e02c0789-64b7-4bb9-9f03-f270c158eb9f req-2f5120d6-b503-469c-94de-48b958997f0e tempest-VolumesSnapshotsTestJSON-21813657 None] Lock "b'cinder-06b9663d-74c3-44b0-835e-37f8e32120ba-delete_snapshot'" released by "delete_snapshot" :: held 3.654s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:13:43.020271 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] Lock "b'cinder-290d5498-182e-4bc7-bcec-0cf0a2ec578a-delete_volume'" acquired by "delete_volume" :: waited 0.032s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:13:43.225096 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:43.594596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:13:43.596027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] Volume volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:13:43.776754 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] deleting rbd volume volume-290d5498-182e-4bc7-bcec-0cf0a2ec578a {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:13:50.256068 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] Created reservations ['301c7cff-3b3e-46a2-bcbf-a9e504241b5d', '1e973072-bb61-4e86-b780-61d50d3186ae', '84df3588-6873-425c-a8a1-ec54608d7867', 'bb28e376-57ac-47d2-9e53-f5b26f3a9bf9'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:13:50.512308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:50.589406 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:13:50.643561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:13:50.648630 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] Deleted volume successfully. May 28 21:13:50.738624 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-66906741-a04f-466d-87be-1d82172ffd6e req-15088536-da7d-45f5-a29c-28b69bbb7fa8 tempest-VolumesSnapshotsTestJSON-21813657 None] Lock "b'cinder-290d5498-182e-4bc7-bcec-0cf0a2ec578a-delete_volume'" released by "delete_volume" :: held 7.719s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:13:58.277442 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-bf4839b4-eb97-4227-95e4-1d41a1d482b1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:13:59.270988 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-bf4839b4-eb97-4227-95e4-1d41a1d482b1 tempest-TaggedBootDevicesTest_v242-1600206585 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.993s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:13:59.271936 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-bf4839b4-eb97-4227-95e4-1d41a1d482b1 tempest-TaggedBootDevicesTest_v242-1600206585 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-26cabed5-9ce1-42aa-b829-f68d323905f3', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '26cabed5-9ce1-42aa-b829-f68d323905f3', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:13:59.358390 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-bf4839b4-eb97-4227-95e4-1d41a1d482b1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Connection info returned from driver {'name': 'volumes/volume-26cabed5-9ce1-42aa-b829-f68d323905f3', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '26cabed5-9ce1-42aa-b829-f68d323905f3', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '8214d51c-c188-42b7-a71c-00823290c271'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:13:59.514800 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-bf4839b4-eb97-4227-95e4-1d41a1d482b1 tempest-TaggedBootDevicesTest_v242-1600206585 None] attachment_update completed successfully. May 28 21:14:00.198106 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-a02b8345-a3e7-42a1-84e4-7cef815d002d tempest-TaggedBootDevicesTest_v242-1600206585 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:14:00.795624 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:14:00.797491 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:14:00.835279 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:14:00.866347 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:14:01.123369 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-a02b8345-a3e7-42a1-84e4-7cef815d002d tempest-TaggedBootDevicesTest_v242-1600206585 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.926s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:14:01.124463 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-a02b8345-a3e7-42a1-84e4-7cef815d002d tempest-TaggedBootDevicesTest_v242-1600206585 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-50730637-3c48-4270-bed2-229c25f4f463', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '50730637-3c48-4270-bed2-229c25f4f463', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:14:01.227444 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-a02b8345-a3e7-42a1-84e4-7cef815d002d tempest-TaggedBootDevicesTest_v242-1600206585 None] Connection info returned from driver {'name': 'volumes/volume-50730637-3c48-4270-bed2-229c25f4f463', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '50730637-3c48-4270-bed2-229c25f4f463', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'd9d2e024-3f33-45e9-8e34-07d8213df050'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:14:01.379444 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-a02b8345-a3e7-42a1-84e4-7cef815d002d tempest-TaggedBootDevicesTest_v242-1600206585 None] attachment_update completed successfully. May 28 21:14:02.517019 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=48,cinder:UPDATE=18,cinder:DELETE=3,cinder:INSERT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:02.521733 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-0c081785-9224-469c-95f1-6a6252fea96b tempest-TaggedBootDevicesTest_v242-1600206585 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:14:03.358916 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-0c081785-9224-469c-95f1-6a6252fea96b tempest-TaggedBootDevicesTest_v242-1600206585 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.838s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:14:03.359822 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-0c081785-9224-469c-95f1-6a6252fea96b tempest-TaggedBootDevicesTest_v242-1600206585 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-0aae4e20-4ac4-4c87-a853-064fb70796fb', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '0aae4e20-4ac4-4c87-a853-064fb70796fb', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:14:03.363375 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=64,cinder:UPDATE=23,cinder:DELETE=1,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:03.523172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-0c081785-9224-469c-95f1-6a6252fea96b tempest-TaggedBootDevicesTest_v242-1600206585 None] Connection info returned from driver {'name': 'volumes/volume-0aae4e20-4ac4-4c87-a853-064fb70796fb', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '0aae4e20-4ac4-4c87-a853-064fb70796fb', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0310d57d-44fa-4897-a2a7-e460e5c53cca'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:14:03.651625 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-b52fc8ad-9c98-49be-acc2-58caeeb15176 req-0c081785-9224-469c-95f1-6a6252fea96b tempest-TaggedBootDevicesTest_v242-1600206585 None] attachment_update completed successfully. May 28 21:14:21.668908 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=17,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:41.689229 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:14:59.039238 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] Lock "b'cinder-97e7aab3-d275-463e-aaa5-5830a8126a8d-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:14:59.187967 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:14:59.453087 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:14:59.453591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] Volume volume-97e7aab3-d275-463e-aaa5-5830a8126a8d is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:14:59.819432 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] deleting rbd volume volume-97e7aab3-d275-463e-aaa5-5830a8126a8d {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:15:00.793917 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:15:00.806205 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:15:00.865252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:15:00.933614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:15:06.735093 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=13,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:06.801244 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] Created reservations ['5ec972f4-0429-4ded-801c-9d4e5bc61500', '9b6306fa-0897-41c8-be0b-ef330de58532', 'd72ab0ef-66f2-43de-b1fa-0403043c96d5', '3edc251e-20e0-41da-be8c-af7ce2f0f432'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:15:07.082248 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:15:07.106023 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:15:07.133619 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:15:07.138723 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] Deleted volume successfully. May 28 21:15:07.262885 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-25e6debb-68e0-46e4-839d-ced13bf5eb30 tempest-DeleteServersTestJSON-435436475 None] Lock "b'cinder-97e7aab3-d275-463e-aaa5-5830a8126a8d-delete_volume'" released by "delete_volume" :: held 8.224s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:15:31.699918 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=23,cinder:UPDATE=17,cinder:INSERT=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:15:32.605623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:15:32.639228 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Flow 'volume_create_manager' (fb12cb23-637e-4d47-94c1-4e13828ac714) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:15:32.647112 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19164290-cfee-499c-b3d4-8f47d68dc8bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:15:32.678282 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19164290-cfee-499c-b3d4-8f47d68dc8bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:15:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-661608883',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=63775810-6872-4bc9-9e90-435936e0987e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='53b9cfacdbb74a058e52bed331a6f241',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:15:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:15:32Z,use_quota=True,user_id='af446b3d802a42e3bd72b08bace8ea8d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:15:32.688534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ba6bb7ca-1ecf-4057-aa32-773a5efd254a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:15:32.697942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ba6bb7ca-1ecf-4057-aa32-773a5efd254a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:15:32.703269 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f1f5493-e19f-40f0-9b39-f2f9de2a2bc8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:15:32.707958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4f1f5493-e19f-40f0-9b39-f2f9de2a2bc8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '63775810-6872-4bc9-9e90-435936e0987e', 'volume_name': 'volume-63775810-6872-4bc9-9e90-435936e0987e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:15:32.713185 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6b72fa00-145d-48c3-af31-ad066a00ed25) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:15:32.717421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6b72fa00-145d-48c3-af31-ad066a00ed25) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:15:32.722698 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c4c11da3-fee5-41db-85a8-1cada571ce3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:15:32.724322 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume 63775810-6872-4bc9-9e90-435936e0987e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-63775810-6872-4bc9-9e90-435936e0987e', 'volume_size': 1} May 28 21:15:32.725162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] creating volume 'volume-63775810-6872-4bc9-9e90-435936e0987e' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:15:32.726703 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:15:34.304876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:15:34.305430 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:15:34.310539 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c4c11da3-fee5-41db-85a8-1cada571ce3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-63775810-6872-4bc9-9e90-435936e0987e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:15:34.321732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1464f101-8e68-4924-b745-cc8f2340ef08) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:15:34.465779 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume volume-63775810-6872-4bc9-9e90-435936e0987e (63775810-6872-4bc9-9e90-435936e0987e): created successfully May 28 21:15:34.470989 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1464f101-8e68-4924-b745-cc8f2340ef08) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:15:34.490565 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Flow 'volume_create_manager' (fb12cb23-637e-4d47-94c1-4e13828ac714) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:15:34.581684 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-be4e15dc-6011-4550-be3e-5476e668016d tempest-ServerRescueNegativeTestJSON-1405524112 None] Created volume successfully. May 28 21:15:41.708403 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:DELETE=1,cinder:SELECT=13,cinder:UPDATE=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:00.792797 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:16:00.793415 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:16:00.875013 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:16:00.961149 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:16:04.078140 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:16:04.099448 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Flow 'volume_create_manager' (4a2ea842-11a2-4e00-8237-00f262d49b93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:16:04.107160 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7413eeff-daec-4122-aa61-06f601f31dc4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:16:04.126863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7413eeff-daec-4122-aa61-06f601f31dc4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:16:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerRescueNegativeTestJSON-volume-612385192',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=91db7a6e-9511-44a8-bcd6-90cbdc41f7f0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='53b9cfacdbb74a058e52bed331a6f241',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:16:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:16:03Z,use_quota=True,user_id='af446b3d802a42e3bd72b08bace8ea8d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:16:04.134612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0d3b263b-13d3-4d0e-b6e3-0e6531a012b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:16:04.140131 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0d3b263b-13d3-4d0e-b6e3-0e6531a012b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:16:04.145863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (04cad695-e7d9-44ff-8ab9-97f082da6016) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:16:04.150748 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (04cad695-e7d9-44ff-8ab9-97f082da6016) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'volume_name': 'volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:16:04.155867 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f4dcd0f6-6b86-48d8-9f60-6f95841ef50d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:16:04.160231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f4dcd0f6-6b86-48d8-9f60-6f95841ef50d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:16:04.167203 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (30fdc12d-0ba6-4da6-b07f-8a1ae068272c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:16:04.168798 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume 91db7a6e-9511-44a8-bcd6-90cbdc41f7f0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'volume_size': 1} May 28 21:16:04.169398 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] creating volume 'volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:16:04.170287 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:16:05.191048 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:16:05.191728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:16:05.195253 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (30fdc12d-0ba6-4da6-b07f-8a1ae068272c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:16:05.200226 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7c0ed195-0423-4979-a9ef-2a57d8f807df) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:16:05.207497 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=11,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:05.337102 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0 (91db7a6e-9511-44a8-bcd6-90cbdc41f7f0): created successfully May 28 21:16:05.345633 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7c0ed195-0423-4979-a9ef-2a57d8f807df) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:16:05.357626 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Flow 'volume_create_manager' (4a2ea842-11a2-4e00-8237-00f262d49b93) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:16:05.440653 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e1003592-d609-4751-8df3-1ff51d9e603b tempest-ServerRescueNegativeTestJSON-1405524112 None] Created volume successfully. May 28 21:16:21.690838 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=11,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:41.695858 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:16:51.701540 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:00.796927 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:17:00.798341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:17:00.863164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:17:00.939840 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:17:05.917783 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c68549e6-e894-4d4f-b402-13ceecb328da req-ad72954c-2838-4337-b0dd-c9357b2a7518 tempest-ServerRescueNegativeTestJSON-1405524112 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:17:06.629251 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c68549e6-e894-4d4f-b402-13ceecb328da req-ad72954c-2838-4337-b0dd-c9357b2a7518 tempest-ServerRescueNegativeTestJSON-1405524112 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.711s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:17:06.630419 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c68549e6-e894-4d4f-b402-13ceecb328da req-ad72954c-2838-4337-b0dd-c9357b2a7518 tempest-ServerRescueNegativeTestJSON-1405524112 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:17:06.755471 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c68549e6-e894-4d4f-b402-13ceecb328da req-ad72954c-2838-4337-b0dd-c9357b2a7518 tempest-ServerRescueNegativeTestJSON-1405524112 None] Connection info returned from driver {'name': 'volumes/volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '91db7a6e-9511-44a8-bcd6-90cbdc41f7f0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'af6d9944-a6b9-4fa5-8aae-d1d0474501ca'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:17:06.924766 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c68549e6-e894-4d4f-b402-13ceecb328da req-ad72954c-2838-4337-b0dd-c9357b2a7518 tempest-ServerRescueNegativeTestJSON-1405524112 None] attachment_update completed successfully. May 28 21:17:21.701693 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=17,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:31.704812 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=16,cinder:UPDATE=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:17:51.714272 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:00.794630 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:18:00.796182 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:00.861768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:00.948620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:18:06.937331 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:06.943609 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-e8ccabc3-122c-480c-88af-2b7d16bd0e52 tempest-ServersTestBootFromVolume-78716756 None] Terminate volume connection completed successfully. May 28 21:18:06.944030 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-e8ccabc3-122c-480c-88af-2b7d16bd0e52 tempest-ServersTestBootFromVolume-78716756 None] Deleting attachment eaa9fe51-c32a-4ac0-8e73-abc037744cc6. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:18:07.734391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] Lock "b'cinder-305738dc-96e3-44a1-968b-a25b737a3d47-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:18:07.907830 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:08.213727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:18:08.503229 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] deleting rbd volume volume-305738dc-96e3-44a1-968b-a25b737a3d47 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:18:16.971460 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-fb5a6227-aa46-4cbd-ac7f-5b355d4c8065 req-91ad636f-ddc4-4b42-a4fa-c0b440288e89 tempest-ServerRescueNegativeTestJSON-1405524112 None] Terminate volume connection completed successfully. May 28 21:18:16.972691 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-fb5a6227-aa46-4cbd-ac7f-5b355d4c8065 req-91ad636f-ddc4-4b42-a4fa-c0b440288e89 tempest-ServerRescueNegativeTestJSON-1405524112 None] Deleting attachment af6d9944-a6b9-4fa5-8aae-d1d0474501ca. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:18:18.432220 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:18:18.754388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] deleting parent snapshot volume-305738dc-96e3-44a1-968b-a25b737a3d47.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:18:23.110420 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=16,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:23.289979 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] Created reservations ['cb51cfa8-ef36-4d70-a043-72a77e8c453e', 'd2415642-f61c-47c2-be88-f3027e4cfa74', 'a1a72fe3-7c9e-4560-8e5a-85aca3f398f2', 'c9315477-90d0-498d-8922-27b0c7cce204'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:18:23.558121 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:23.617423 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:23.700251 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:18:23.706494 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] Deleted volume successfully. May 28 21:18:23.854716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-2a61c647-0153-4c44-bce1-4096f4a87a58 req-de03c126-7d24-4dc1-9187-1c113eb12bfd tempest-ServersTestBootFromVolume-78716756 None] Lock "b'cinder-305738dc-96e3-44a1-968b-a25b737a3d47-delete_volume'" released by "delete_volume" :: held 16.125s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:18:29.244735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] Lock "b'cinder-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:18:29.391983 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:29.602940 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:18:29.603700 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:18:29.985487 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] deleting rbd volume volume-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:18:39.162956 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] Created reservations ['5223266e-257c-40e8-80be-12bb3a3140c9', '6693cabe-0f0d-470c-8814-e980e9e5357b', 'fb728e4b-dad0-43d8-b084-ca5d189e3314', 'f1b8fcd1-e0fe-43d8-853f-fcd8027a7b16'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:18:39.352027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:39.413219 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:39.466293 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:18:39.473177 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] Deleted volume successfully. May 28 21:18:39.476773 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=41,cinder:UPDATE=27,cinder:INSERT=8,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:18:39.559145 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f36aba5d-1441-41c5-8a34-4275bad6e812 tempest-ServerRescueNegativeTestJSON-1405524112 None] Lock "b'cinder-91db7a6e-9511-44a8-bcd6-90cbdc41f7f0-delete_volume'" released by "delete_volume" :: held 10.313s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:18:40.445146 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] Lock "b'cinder-63775810-6872-4bc9-9e90-435936e0987e-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:18:40.558855 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:40.833276 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:18:40.833913 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] Volume volume-63775810-6872-4bc9-9e90-435936e0987e is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:18:41.089096 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] deleting rbd volume volume-63775810-6872-4bc9-9e90-435936e0987e {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:18:48.974620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] Created reservations ['0119bea8-1f5b-4a2a-b3b5-b49ecb83d225', 'bcb0bded-9d8c-469b-afee-eb613f3bd135', 'c7cd3d40-bbea-48d5-98cf-724f80e8d938', '5dfac61a-62b7-44d5-90ba-3533de1cdecc'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:18:49.282297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:49.391170 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:18:49.461670 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:18:49.467934 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] Deleted volume successfully. May 28 21:18:49.562797 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9aab76c2-40a0-4017-ba8e-0347e870e4b1 tempest-ServerRescueNegativeTestJSON-1405524112 None] Lock "b'cinder-63775810-6872-4bc9-9e90-435936e0987e-delete_volume'" released by "delete_volume" :: held 9.116s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:19:00.797458 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:19:00.799265 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:00.863187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:00.953151 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:19:03.109236 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=29,cinder:UPDATE=17,cinder:DELETE=1,cinder:INSERT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:13.102677 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=16,cinder:INSERT=1,cinder:UPDATE=9,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:26.762518 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:19:26.783889 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Flow 'volume_create_manager' (4ee2dda3-e3fe-4f79-818e-fd2f82127f22) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:19:26.791174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e241136-42e7-48e8-af6f-cf861271276c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:26.811187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6e241136-42e7-48e8-af6f-cf861271276c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:19:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeRetypeWithoutMigrationTest-Volume-731241209',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2973fd62-1c93-4b3e-ab76-52e1bfc35639,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0f6041dc987549cfb48003c2b424ada5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:19:26Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:19:27Z,use_quota=True,user_id='661333ccdbc74a96ac12d051ad3c2279',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(6422ea91-c683-4898-a5d9-a701fe6e9d00),volume_type_id=6422ea91-c683-4898-a5d9-a701fe6e9d00)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:26.816681 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c3bd809c-498b-40db-b575-daa227b51b3b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:26.821170 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c3bd809c-498b-40db-b575-daa227b51b3b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:26.826412 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9eae4de-b815-470f-b1bf-b9b46b22a94e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:26.830933 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9eae4de-b815-470f-b1bf-b9b46b22a94e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '2973fd62-1c93-4b3e-ab76-52e1bfc35639', 'volume_name': 'volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:26.836228 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7c309fdc-3277-4708-bbdf-4dabb243e787) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:26.840830 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7c309fdc-3277-4708-bbdf-4dabb243e787) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:26.846466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b6742071-d9bb-4cba-a9fc-ba45880358b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:26.847818 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Volume 2973fd62-1c93-4b3e-ab76-52e1bfc35639: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639', 'volume_size': 1} May 28 21:19:26.848435 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] creating volume 'volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:19:26.849392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:28.370495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:19:28.372424 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:19:28.378855 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b6742071-d9bb-4cba-a9fc-ba45880358b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:28.386286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d7570013-3f59-4918-88a5-243b3206a272) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:28.526345 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Volume volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639 (2973fd62-1c93-4b3e-ab76-52e1bfc35639): created successfully May 28 21:19:28.533636 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d7570013-3f59-4918-88a5-243b3206a272) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:28.549060 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Flow 'volume_create_manager' (4ee2dda3-e3fe-4f79-818e-fd2f82127f22) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:19:28.674819 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-aee3c49b-e76d-41c5-8127-e4259b8836be tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Created volume successfully. May 28 21:19:30.995117 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a74586a-6673-41d5-a7d7-c67199c645d6 tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:31.069343 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a74586a-6673-41d5-a7d7-c67199c645d6 tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:31.161418 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-2a74586a-6673-41d5-a7d7-c67199c645d6 tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:19:31.188309 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2a74586a-6673-41d5-a7d7-c67199c645d6 tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Retype volume completed successfully. May 28 21:19:31.196589 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-84092fe4-2f98-46a5-bbcc-598488638864 tempest-GroupsV314Test-2072414033 None] Group tempest-GroupsV314Test-Group-2133267787: creating May 28 21:19:31.303662 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-84092fe4-2f98-46a5-bbcc-598488638864 tempest-GroupsV314Test-2072414033 None] Group tempest-GroupsV314Test-Group-2133267787: created successfully May 28 21:19:31.308851 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-84092fe4-2f98-46a5-bbcc-598488638864 tempest-GroupsV314Test-2072414033 None] Create group completed successfully. May 28 21:19:32.217773 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Lock "b'cinder-2973fd62-1c93-4b3e-ab76-52e1bfc35639-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:19:32.337584 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:32.560447 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:19:32.561144 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Volume volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:19:32.820047 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] deleting rbd volume volume-2973fd62-1c93-4b3e-ab76-52e1bfc35639 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:19:33.680545 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:19:33.712086 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Flow 'volume_create_manager' (417f1529-9f05-4443-9530-1b81078477a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:19:33.726985 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9696d689-ec68-4653-9559-09df9d5564c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:33.752426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9696d689-ec68-4653-9559-09df9d5564c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:19:33Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsV314Test-Volume-391975204',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=22adfadf-229b-482b-bf77-fd62df836ea8,host='ncdv-master-1@ceph#ceph',id=ee9b7283-df57-44e2-a715-ef87de444dc6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='530a8242ae014464b176c523ae99294e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:19:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:19:33Z,use_quota=True,user_id='cfa39677553343bcaf62286df0ec6379',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(8bb63010-7749-4774-a2e8-106e47207973),volume_type_id=8bb63010-7749-4774-a2e8-106e47207973)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:33.763084 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3c3b795f-e4d8-40bc-a85d-951149232e21) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:33.768624 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3c3b795f-e4d8-40bc-a85d-951149232e21) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:33.781729 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (83fda605-687a-4a8d-936d-143b602f67bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:33.786675 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (83fda605-687a-4a8d-936d-143b602f67bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ee9b7283-df57-44e2-a715-ef87de444dc6', 'volume_name': 'volume-ee9b7283-df57-44e2-a715-ef87de444dc6', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:33.791943 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (73d05f39-5490-41ec-b802-727e52b1d9e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:33.802825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (73d05f39-5490-41ec-b802-727e52b1d9e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:33.821938 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6543e0be-d9cb-4dc3-b3b4-650590c283ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:33.823394 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Volume ee9b7283-df57-44e2-a715-ef87de444dc6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ee9b7283-df57-44e2-a715-ef87de444dc6', 'volume_size': 1} May 28 21:19:33.823789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] creating volume 'volume-ee9b7283-df57-44e2-a715-ef87de444dc6' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:19:33.824450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:34.668372 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:19:34.673252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:19:34.673602 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6543e0be-d9cb-4dc3-b3b4-650590c283ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ee9b7283-df57-44e2-a715-ef87de444dc6', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:34.681014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7866123f-fa54-4cf5-9d90-34295daf0451) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:34.789586 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Volume volume-ee9b7283-df57-44e2-a715-ef87de444dc6 (ee9b7283-df57-44e2-a715-ef87de444dc6): created successfully May 28 21:19:34.793180 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7866123f-fa54-4cf5-9d90-34295daf0451) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:34.802056 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Flow 'volume_create_manager' (417f1529-9f05-4443-9530-1b81078477a8) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:19:34.859364 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0ce91026-618e-48ef-bb32-635a30f892a2 tempest-GroupsV314Test-2072414033 None] Created volume successfully. May 28 21:19:36.735945 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=28,cinder:UPDATE=15,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:36.776680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:37.006268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] creating snapshot='volume-89db329e-c1e9-48d6-8cfc-5527b838ce44.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:19:39.726479 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] cloning 'volume-ee9b7283-df57-44e2-a715-ef87de444dc6@volume-89db329e-c1e9-48d6-8cfc-5527b838ce44.clone_snap' to 'volume-89db329e-c1e9-48d6-8cfc-5527b838ce44' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:19:39.761080 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Deprecated: Default quota for resource: volumes_volume-type-2 is set by the default quota flag: quota_volumes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. May 28 21:19:39.761931 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Deprecated: Default quota for resource: gigabytes_volume-type-2 is set by the default quota flag: quota_gigabytes_volume-type-2, it is now deprecated. Please use the default quota class for default quota. May 28 21:19:39.944412 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Created reservations ['85acc9f7-4b5b-4b21-8ffd-a2dff5950291', '163cef77-9e31-45a9-8724-dee943bc2093', 'c65457bc-3030-4314-a02f-6fe4d960405f', '3cdec1e1-6d5e-4ac9-a9a7-759055b6bd58'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:19:40.449401 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:40.531962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:40.588466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:19:40.594037 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Deleted volume successfully. May 28 21:19:40.782084 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9a032467-9c8c-43a1-ac3e-7ab4e894016b tempest-VolumeRetypeWithoutMigrationTest-1874840680 None] Lock "b'cinder-2973fd62-1c93-4b3e-ab76-52e1bfc35639-delete_volume'" released by "delete_volume" :: held 8.559s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:19:41.811749 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] Volume volume-ee9b7283-df57-44e2-a715-ef87de444dc6 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:19:41.824036 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:19:41.826825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:19:42.287488 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:19:42.753048 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=70,cinder:UPDATE=24,cinder:DELETE=2,cinder:INSERT=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:42.758220 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-50a56c61-0f42-4465-b9b0-44feac01ffbe tempest-GroupsV314Test-2072414033 None] Create group from source-group-22adfadf-229b-482b-bf77-fd62df836ea8 completed successfully. May 28 21:19:43.496888 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:43.914787 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:19:44.116562 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] deleting rbd volume volume-89db329e-c1e9-48d6-8cfc-5527b838ce44 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:19:50.222208 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:19:50.434809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Volume volume-ee9b7283-df57-44e2-a715-ef87de444dc6 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:19:50.435406 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] deleting parent snapshot volume-89db329e-c1e9-48d6-8cfc-5527b838ce44.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:19:53.830292 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=11,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:19:54.050954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:19:54.102962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Flow 'volume_create_manager' (0d1564b9-d389-4d06-9832-d69f4058d09f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:19:54.110174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (84395f8e-366c-4892-a042-4d716b5dfba3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:54.148275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (84395f8e-366c-4892-a042-4d716b5dfba3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:19:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeSnapshotQuotasNegativeTestJSON-Volume-13521475',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b08c9cab-8904-4ea0-bc09-e79f971e9fd0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c76d04ad120d41f6a3bfabb87bf6968e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:19:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:19:51Z,use_quota=True,user_id='d44fb59970fb442da24187a6ec827b75',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:54.154213 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5792e90e-2172-4966-b072-66a767584be8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:54.159699 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5792e90e-2172-4966-b072-66a767584be8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:54.165381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4714c886-4da2-4ebf-bd6a-8e63f6130ee8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:54.173488 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4714c886-4da2-4ebf-bd6a-8e63f6130ee8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b08c9cab-8904-4ea0-bc09-e79f971e9fd0', 'volume_name': 'volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:54.183727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8098bc9b-9b86-405b-a2f7-1330c34349d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:54.187755 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8098bc9b-9b86-405b-a2f7-1330c34349d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:54.192506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (71ce6e8c-f8dc-4fef-b2da-b866b15ced18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:54.193670 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Volume b08c9cab-8904-4ea0-bc09-e79f971e9fd0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0', 'volume_size': 1} May 28 21:19:54.194081 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] creating volume 'volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:19:54.195039 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:54.344838 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Created reservations ['9bde73b4-ef6e-4ade-aae9-3105c5f2d5dc'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:19:54.389120 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Deprecated: Default quota for resource: volumes_tempest-GroupsV314Test-volume-type-61980273 is set by the default quota flag: quota_volumes_tempest-GroupsV314Test-volume-type-61980273, it is now deprecated. Please use the default quota class for default quota. May 28 21:19:54.393354 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsV314Test-volume-type-61980273 is set by the default quota flag: quota_gigabytes_tempest-GroupsV314Test-volume-type-61980273, it is now deprecated. Please use the default quota class for default quota. May 28 21:19:54.623127 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Created reservations ['4360777c-aa09-4bfb-accc-1510468d68d8', '7eebd989-2950-46f3-be3e-53d389571ddb', '8c12dbd8-29e0-45c6-9923-a0c82c44f69f', 'ed01ad9f-61b0-4c34-a7fb-9a23e69b93e7'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:19:54.945671 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:19:54.950572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:19:54.950572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (71ce6e8c-f8dc-4fef-b2da-b866b15ced18) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:54.959012 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cbadf76a-b47a-493a-97ea-9f9ec06799bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:19:55.057767 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Volume volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0 (b08c9cab-8904-4ea0-bc09-e79f971e9fd0): created successfully May 28 21:19:55.064080 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cbadf76a-b47a-493a-97ea-9f9ec06799bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:19:55.070391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Flow 'volume_create_manager' (0d1564b9-d389-4d06-9832-d69f4058d09f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:19:55.095187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:55.184693 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:55.310218 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:19:55.317496 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a534bcbe-d9f9-4506-b859-4ae06bae5165 tempest-GroupsV314Test-2072414033 None] Delete group completed successfully. May 28 21:19:55.319583 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-3b9ae9e4-cfb5-4157-867f-7402ef73c1ed tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Created volume successfully. May 28 21:19:56.382415 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:56.784415 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:19:56.785562 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] Volume volume-ee9b7283-df57-44e2-a715-ef87de444dc6 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:19:56.903811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] deleting rbd volume volume-ee9b7283-df57-44e2-a715-ef87de444dc6 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:19:56.927317 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-112271af-3d8f-4759-9f1b-0c70ac955018 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:19:59.955992 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-112271af-3d8f-4759-9f1b-0c70ac955018 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Create snapshot completed successfully May 28 21:20:00.800672 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:20:00.801351 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:00.858005 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:00.917623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:20:01.488827 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a9cf90f5-d554-4cb2-bc8e-b8d31c666d21 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Lock "b'cinder-7b52314d-fbe3-4fb2-b803-8755e3189513-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.022s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:01.490354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9cf90f5-d554-4cb2-bc8e-b8d31c666d21 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:03.990076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] Created reservations ['7e5b3e74-dc21-4f60-96f6-ce950ebd9842'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:04.166062 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] Created reservations ['aa855ed4-139f-4783-b19e-5c6d42b7f96e', '93a8521e-32ab-4672-9c8a-1be7eec26006', '54044d6e-c473-4dde-bc4c-ddf0ebce8e7b', '0265549d-51e7-4c29-97fb-e1d2431f9cc8'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:04.181645 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=59,cinder:UPDATE=32,cinder:INSERT=8,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:04.565930 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:04.705797 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:04.733875 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:20:04.738012 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-04516b93-6745-433b-b56a-f117f010fe84 tempest-GroupsV314Test-2072414033 None] Delete group completed successfully. May 28 21:20:06.189179 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a9cf90f5-d554-4cb2-bc8e-b8d31c666d21 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Created reservations ['02c14e9e-dbb2-4642-b343-6dc18bd7c4a4', '78a19cdc-fd30-4182-bf85-48b11b528b03', '4379340c-78a8-41ad-83c1-d5d43e6f768a', '4ce02330-ca7a-497b-9a71-b47c7e426a66'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:06.503516 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a9cf90f5-d554-4cb2-bc8e-b8d31c666d21 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Delete snapshot completed successfully. May 28 21:20:06.510342 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a9cf90f5-d554-4cb2-bc8e-b8d31c666d21 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Lock "b'cinder-7b52314d-fbe3-4fb2-b803-8755e3189513-delete_snapshot'" released by "delete_snapshot" :: held 5.021s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:06.940131 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Lock "b'cinder-b08c9cab-8904-4ea0-bc09-e79f971e9fd0-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:07.139319 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:07.308111 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:20:07.308783 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Volume volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:20:07.646295 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] deleting rbd volume volume-b08c9cab-8904-4ea0-bc09-e79f971e9fd0 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:20:12.824082 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:20:12.843570 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Flow 'volume_create_manager' (46b6e0ec-f3e2-47e5-bfb0-14836328fabd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:12.856334 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0cae98fe-0535-4184-9129-42f1c698409d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:12.856334 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=64,cinder:DELETE=1,cinder:UPDATE=29,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:12.869514 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0cae98fe-0535-4184-9129-42f1c698409d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:20:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotsActionsTest-Volume-1426759246',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a9fba3fa-3d77-4070-8b79-8adf9ed6bedd,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='05724d255c4d44eebb233af520a321f3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:20:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:20:13Z,use_quota=True,user_id='c6817a6607aa48218bea76d56b0eb1f0',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:12.876169 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (77c7f736-e201-4957-afbb-ffed10b52450) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:12.880787 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (77c7f736-e201-4957-afbb-ffed10b52450) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:12.886238 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23ec3239-3f72-4e49-a1f3-ff242ae9eee6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:12.890356 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (23ec3239-3f72-4e49-a1f3-ff242ae9eee6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a9fba3fa-3d77-4070-8b79-8adf9ed6bedd', 'volume_name': 'volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:12.900713 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (76f695a6-baf1-49e4-a316-377e81f864ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:12.907264 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (76f695a6-baf1-49e4-a316-377e81f864ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:12.913137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b88b6b80-834c-44ee-b446-2b04ee0392ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:12.913405 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Volume a9fba3fa-3d77-4070-8b79-8adf9ed6bedd: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd', 'volume_size': 1} May 28 21:20:12.913658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] creating volume 'volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:20:12.914453 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:13.766258 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:20:13.766662 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:20:13.770651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b88b6b80-834c-44ee-b446-2b04ee0392ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:13.775290 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7e44df7f-62d2-4f76-82c7-0af24afd70d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:13.873582 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Volume volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd (a9fba3fa-3d77-4070-8b79-8adf9ed6bedd): created successfully May 28 21:20:13.878850 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7e44df7f-62d2-4f76-82c7-0af24afd70d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:13.882957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Flow 'volume_create_manager' (46b6e0ec-f3e2-47e5-bfb0-14836328fabd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:13.956035 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-562020d4-c2a6-4af7-81c6-768c7b4123c4 tempest-SnapshotsActionsTest-1795257975 None] Created volume successfully. May 28 21:20:14.148642 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Created reservations ['871e9a67-f0ee-4883-9eb2-29982272df37', '51cc863f-8471-450b-b43b-207a06ea0f42', '0de0e768-52e9-4765-bcbc-bd7f2e8b821e', 'd77023b0-df04-4583-bb54-ad9ce23497c1'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:14.379876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:14.412824 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:14.438744 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:20:14.443535 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Deleted volume successfully. May 28 21:20:14.570506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-79bc084e-45a2-4c5c-906f-df51020f6ad6 tempest-VolumeSnapshotQuotasNegativeTestJSON-536968200 None] Lock "b'cinder-b08c9cab-8904-4ea0-bc09-e79f971e9fd0-delete_volume'" released by "delete_volume" :: held 7.630s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:15.361454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-75c15c3b-3a7b-43e0-903f-04aafff158ec tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:18.433908 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-75c15c3b-3a7b-43e0-903f-04aafff158ec tempest-SnapshotsActionsTest-1795257975 None] Create snapshot completed successfully May 28 21:20:20.658124 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-016e47ff-2f94-4059-882f-1b1383873cfd tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:21.755475 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:20:21.778792 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Flow 'volume_create_manager' (18f301e4-caa9-413f-ac2e-96f53dddfa9b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:21.785216 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e6a84c5-4304-48d8-98ee-79067671853c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:21.807381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9e6a84c5-4304-48d8-98ee-79067671853c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:20:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesImageMetadata-Volume-25468428',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='72d4e610a6d641ae9c2c94be6de7de36',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:20:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:20:22Z,use_quota=True,user_id='7bfaf45cd74a4e148f2661c249869977',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:21.812739 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f52bca3e-dab3-406f-bbb7-cb5b82d910e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:21.817199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f52bca3e-dab3-406f-bbb7-cb5b82d910e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:21.822225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (807228a6-c34e-4ddd-918b-e4d54884db96) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:21.923154 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (807228a6-c34e-4ddd-918b-e4d54884db96) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7', 'volume_name': 'volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:21.929164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5bfce996-eac4-46c5-9b7a-6c4c2c37809e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:21.934213 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5bfce996-eac4-46c5-9b7a-6c4c2c37809e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:21.939106 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (58e792ea-a4ed-47ef-b4b6-6c13fced552e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:21.940805 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Volume 5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:20:21.941352 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Cloning 5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:20:21.959329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.017s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:22.044838 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 20, 21, 963727)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:20:22.049720 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:20:22.049720 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:20:22.050698 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.091s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:22.051660 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:20:22.064076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 20, 22, 56462)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:20:22.065037 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:20:22.066105 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:20:22.085031 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:22.301952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] creating snapshot='volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:20:25.228239 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7.clone_snap' to 'volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:20:26.233396 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-016e47ff-2f94-4059-882f-1b1383873cfd tempest-SnapshotsActionsTest-1795257975 None] Create snapshot completed successfully May 28 21:20:27.005901 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-213a474f-a09b-41ce-9259-15a3f5befcc1 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-d620afd0-88d2-4e6e-9d6d-db8ea3325cb6-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.023s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:27.007718 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-213a474f-a09b-41ce-9259-15a3f5befcc1 tempest-SnapshotsActionsTest-1268848655 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:27.372188 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:20:27.373378 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:20:27.373879 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:20:28.026317 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:20:28.027434 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Marking volume 5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:20:28.147329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:20:28.147829 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Creating volume glance metadata for volume 5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:20:28.306549 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (58e792ea-a4ed-47ef-b4b6-6c13fced552e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:28.311330 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c834d663-f15b-4eac-90a9-dcc2fcfbabc4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:28.405092 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Volume volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7 (5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7): created successfully May 28 21:20:28.409233 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c834d663-f15b-4eac-90a9-dcc2fcfbabc4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:28.414795 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Flow 'volume_create_manager' (18f301e4-caa9-413f-ac2e-96f53dddfa9b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:28.469641 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-41f7f8db-b083-463b-ac8a-339d4d0dae0d tempest-VolumesImageMetadata-798598594 None] Created volume successfully. May 28 21:20:29.145627 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:20:29.194534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Flow 'volume_create_manager' (35f7f711-6bdf-427d-a066-a58f17baa6b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:29.214106 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3d1edc0f-06bd-4765-afa5-80e900e9140e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:29.246828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3d1edc0f-06bd-4765-afa5-80e900e9140e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:20:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-1621447547',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=aca16bed-3410-4df8-b98d-9a6854483c48,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5a8841fa441d413a92a15510a9d082c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:20:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:20:29Z,use_quota=True,user_id='857a887b6c7c41588afb2845eed6f309',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:29.254439 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b4c35f0e-48ac-4947-8070-098902adaba2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:29.258894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b4c35f0e-48ac-4947-8070-098902adaba2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:29.267651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5fff593c-1dfd-43e4-8e59-549f4c4406ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:29.271725 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5fff593c-1dfd-43e4-8e59-549f4c4406ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'aca16bed-3410-4df8-b98d-9a6854483c48', 'volume_name': 'volume-aca16bed-3410-4df8-b98d-9a6854483c48', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:29.276815 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (059539ee-2841-4e30-97ba-ed6fdc1f27cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:29.281318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (059539ee-2841-4e30-97ba-ed6fdc1f27cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:29.286590 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (907e5a3f-ee95-415e-85d7-2e3476d04bd1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:29.287788 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume aca16bed-3410-4df8-b98d-9a6854483c48: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-aca16bed-3410-4df8-b98d-9a6854483c48', 'volume_size': 1} May 28 21:20:29.291198 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] creating volume 'volume-aca16bed-3410-4df8-b98d-9a6854483c48' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:20:29.291198 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:29.584849 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] Lock "b'cinder-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:29.738515 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=29,cinder:SELECT=58,cinder:DELETE=5,cinder:INSERT=8 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:29.744620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:30.063228 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:20:30.331630 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] deleting rbd volume volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:20:30.341891 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:20:30.342478 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:20:30.347840 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (907e5a3f-ee95-415e-85d7-2e3476d04bd1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-aca16bed-3410-4df8-b98d-9a6854483c48', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:30.353109 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6302ee78-b82e-4375-83b7-8d70b995397c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:30.430523 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume volume-aca16bed-3410-4df8-b98d-9a6854483c48 (aca16bed-3410-4df8-b98d-9a6854483c48): created successfully May 28 21:20:30.434544 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6302ee78-b82e-4375-83b7-8d70b995397c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:30.440525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Flow 'volume_create_manager' (35f7f711-6bdf-427d-a066-a58f17baa6b7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:30.512701 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-eed3488e-e5d4-4872-8730-a6ecf02e5f81 tempest-AttachVolumeShelveTestJSON-1592071011 None] Created volume successfully. May 28 21:20:30.976484 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-213a474f-a09b-41ce-9259-15a3f5befcc1 tempest-SnapshotsActionsTest-1268848655 None] Created reservations ['adf899ff-710d-47de-b6c2-b89711f0e9dd', '780f5cb9-4ef5-49e4-9b10-1a7eb19ef7b4', '2d5b9925-e028-4031-83c9-a652dcaf377d', 'b6c95828-8294-49a7-b486-e07d7914fa1f'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:31.142797 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-213a474f-a09b-41ce-9259-15a3f5befcc1 tempest-SnapshotsActionsTest-1268848655 None] Delete snapshot completed successfully. May 28 21:20:31.151216 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-213a474f-a09b-41ce-9259-15a3f5befcc1 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-d620afd0-88d2-4e6e-9d6d-db8ea3325cb6-delete_snapshot'" released by "delete_snapshot" :: held 4.146s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:32.151276 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ec432f3f-00af-4512-9d48-7fbaaef8aae2 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:35.227537 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ec432f3f-00af-4512-9d48-7fbaaef8aae2 tempest-SnapshotsActionsTest-1795257975 None] Create snapshot completed successfully May 28 21:20:36.287058 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1bf42b16-7dce-4ef5-aec6-2b03a7f27714 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-3567b76d-f4e6-40df-9fe8-9f7c17b64b95-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.023s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:36.289073 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1bf42b16-7dce-4ef5-aec6-2b03a7f27714 tempest-SnapshotsActionsTest-1268848655 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:36.460646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:20:36.710286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] deleting parent snapshot volume-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:20:39.953444 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] Created reservations ['d88b3478-892c-4e98-9fd5-0f6c9893a697', '4feb5f0b-3b03-424f-8b39-931d81e09ffa', 'f2e3790a-4c4f-40da-ac8e-86a38609c122', '8875afd0-fa4b-4fa6-af64-98dd06915257'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:40.148677 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=61,cinder:UPDATE=24,cinder:INSERT=13,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:40.299543 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:40.354752 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:40.413054 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:20:40.418966 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] Deleted volume successfully. May 28 21:20:40.555733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a4b16b62-54ca-43cf-a1e0-c61ff2d5cff9 tempest-VolumesImageMetadata-798598594 None] Lock "b'cinder-5d135961-8cb2-4c2b-9e9c-e6d93d14c0b7-delete_volume'" released by "delete_volume" :: held 10.968s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:43.109618 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-1bf42b16-7dce-4ef5-aec6-2b03a7f27714 tempest-SnapshotsActionsTest-1268848655 None] Created reservations ['bbe51287-78f0-4c99-8683-587ef9dfa877', '159980a9-210c-44aa-b160-84a470691959', '01d10abb-2d27-4e8a-a553-1127a3859696', '03dcf97a-17fb-41f2-86a7-5a89a6557087'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:43.473130 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1bf42b16-7dce-4ef5-aec6-2b03a7f27714 tempest-SnapshotsActionsTest-1268848655 None] Delete snapshot completed successfully. May 28 21:20:43.479176 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1bf42b16-7dce-4ef5-aec6-2b03a7f27714 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-3567b76d-f4e6-40df-9fe8-9f7c17b64b95-delete_snapshot'" released by "delete_snapshot" :: held 7.191s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:44.630150 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d602fa4-70a7-48d0-9601-1406dd673d82 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:48.654004 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-ccb0393b-dfe1-4030-b45c-87c7211a8a86 req-d711fce4-9968-4adc-8e78-44464548ce44 tempest-TaggedBootDevicesTest-427054523 None] Terminate volume connection completed successfully. May 28 21:20:48.655966 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ccb0393b-dfe1-4030-b45c-87c7211a8a86 req-d711fce4-9968-4adc-8e78-44464548ce44 tempest-TaggedBootDevicesTest-427054523 None] Deleting attachment 4c85200e-398a-4fcf-b339-c25a641cc399. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:20:48.839787 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1d602fa4-70a7-48d0-9601-1406dd673d82 tempest-SnapshotsActionsTest-1795257975 None] Create snapshot completed successfully May 28 21:20:48.962531 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-ccb0393b-dfe1-4030-b45c-87c7211a8a86 req-9e67bd6a-f529-4769-91a2-28779d846eb4 tempest-TaggedBootDevicesTest-427054523 None] Terminate volume connection completed successfully. May 28 21:20:48.963246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ccb0393b-dfe1-4030-b45c-87c7211a8a86 req-9e67bd6a-f529-4769-91a2-28779d846eb4 tempest-TaggedBootDevicesTest-427054523 None] Deleting attachment 494b2820-bd5e-44d5-aadd-27b830314cad. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:20:49.337872 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-ccb0393b-dfe1-4030-b45c-87c7211a8a86 req-c9ba3069-d503-4928-9503-6e16307df425 tempest-TaggedBootDevicesTest-427054523 None] Terminate volume connection completed successfully. May 28 21:20:49.338878 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ccb0393b-dfe1-4030-b45c-87c7211a8a86 req-c9ba3069-d503-4928-9503-6e16307df425 tempest-TaggedBootDevicesTest-427054523 None] Deleting attachment 5c3711d3-fbc9-4589-8036-da112d6f1b65. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:20:50.157170 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2844e5f5-5c9b-497f-8238-766ae679e917 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-c811275b-0e77-463a-b11b-570c0a907c45-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.030s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:20:50.160555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2844e5f5-5c9b-497f-8238-766ae679e917 tempest-SnapshotsActionsTest-1268848655 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:50.812048 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:20:50.865868 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Flow 'volume_create_manager' (8b8d10f5-b1d7-4ace-be27-417d5e4e8b76) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:50.907637 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2173533d-f8a6-4d9a-a267-ae106a7a1978) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:50.957230 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2173533d-f8a6-4d9a-a267-ae106a7a1978) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:20:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1042412530',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='877bd1cd8912437c8e4871df8a5f7260',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:20:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:20:51Z,use_quota=True,user_id='84f18f11aad344c18bb6b3997370576b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:50.967001 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ace9d68b-b9fe-4530-a380-11a612b75d0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:50.981363 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ace9d68b-b9fe-4530-a380-11a612b75d0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:50.990809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7c7c0b8f-d1c2-4ad8-a8b2-a4a3700c39ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:51.000938 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7c7c0b8f-d1c2-4ad8-a8b2-a4a3700c39ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc', 'volume_name': 'volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:51.020996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d3600804-3765-4515-b1d9-c9c599320a14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:51.030950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d3600804-3765-4515-b1d9-c9c599320a14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:51.046597 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fe8ef38f-9480-406c-8c46-229770c03afc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:51.048302 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Volume 367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc', 'volume_size': 1} May 28 21:20:51.048934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] creating volume 'volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:20:51.049707 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:20:52.326944 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:20:52.328126 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:20:52.336057 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fe8ef38f-9480-406c-8c46-229770c03afc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:52.346036 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (19735d49-9b4e-480b-897d-4d53a2b25a77) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:20:52.463297 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Volume volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc (367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc): created successfully May 28 21:20:52.468801 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (19735d49-9b4e-480b-897d-4d53a2b25a77) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:20:52.481565 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Flow 'volume_create_manager' (8b8d10f5-b1d7-4ace-be27-417d5e4e8b76) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:20:52.589030 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-65eb4020-b40f-468b-8ba9-f7f3bded30d9 tempest-VolumesActionsTest-514564619 None] Created volume successfully. May 28 21:20:54.901235 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2844e5f5-5c9b-497f-8238-766ae679e917 tempest-SnapshotsActionsTest-1268848655 None] Created reservations ['f622101f-2d57-4c38-b597-b1855aae2b97', 'ca66c710-8ffc-454d-a702-8421f7588321', 'd1d67fbf-9310-4fc8-b3f1-0d46d7ab44de', 'e6159370-c3c8-41a9-9ae1-f20067a51bc4'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:20:54.905211 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=61,cinder:UPDATE=27,cinder:INSERT=10,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:20:55.271440 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2844e5f5-5c9b-497f-8238-766ae679e917 tempest-SnapshotsActionsTest-1268848655 None] Delete snapshot completed successfully. May 28 21:20:55.278807 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2844e5f5-5c9b-497f-8238-766ae679e917 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-c811275b-0e77-463a-b11b-570c0a907c45-delete_snapshot'" released by "delete_snapshot" :: held 5.119s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:20:57.124145 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f273bc3d-e814-461f-83a2-3e8b0a5dc51d tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:00.803246 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:21:00.803246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:00.882811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:00.943887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:21:01.072510 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f273bc3d-e814-461f-83a2-3e8b0a5dc51d tempest-SnapshotsActionsTest-1795257975 None] Create snapshot completed successfully May 28 21:21:02.376223 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-c7ec9972-c33d-41d5-bf85-a57642501bd3-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:02.482556 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c6453fa5-b450-4bd0-bfec-3ebce3cb3094 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-75f8775a-0e43-4e6d-aed5-53eb62007d97-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.074s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:02.492246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c6453fa5-b450-4bd0-bfec-3ebce3cb3094 tempest-SnapshotsActionsTest-1268848655 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:02.845047 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:03.020740 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:21:03.025466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] Volume volume-c7ec9972-c33d-41d5-bf85-a57642501bd3 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:21:03.279136 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] deleting rbd volume volume-c7ec9972-c33d-41d5-bf85-a57642501bd3 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:21:06.456074 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-c6453fa5-b450-4bd0-bfec-3ebce3cb3094 tempest-SnapshotsActionsTest-1268848655 None] Created reservations ['d16f19f1-86fc-4cbc-8c13-6b3205d30272', 'cd8b3f91-3358-4770-b30b-b1498725dc1e', '4b8a188e-0e1e-4d39-ab44-026ac365be0e', '7dc86ee4-b56f-441a-944a-f4da7bbec980'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:06.769573 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c6453fa5-b450-4bd0-bfec-3ebce3cb3094 tempest-SnapshotsActionsTest-1268848655 None] Delete snapshot completed successfully. May 28 21:21:06.780964 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c6453fa5-b450-4bd0-bfec-3ebce3cb3094 tempest-SnapshotsActionsTest-1268848655 None] Lock "b'cinder-75f8775a-0e43-4e6d-aed5-53eb62007d97-delete_snapshot'" released by "delete_snapshot" :: held 4.298s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:08.544877 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2777e460-0431-437b-bc3f-ac22877e26c0 tempest-SnapshotsActionsTest-1795257975 None] Lock "b'cinder-71d8b4ce-6fa6-40fd-8722-b2f93b50039a-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.018s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:08.547525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2777e460-0431-437b-bc3f-ac22877e26c0 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:10.310717 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] Created reservations ['a0327418-1788-475f-ba1f-43a43f1f8237', '07f52d6c-9c88-49f9-8c18-132257c653bd', '8b14d696-e3b7-4e97-ace4-f4bac18a3439', 'ebdf494c-a4ac-4450-9c6f-bd376c2efe9a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:10.610672 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:10.662955 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:10.711604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:21:10.716007 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] Deleted volume successfully. May 28 21:21:10.806426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d1204e53-9354-4b98-a07a-462c93694590 tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-c7ec9972-c33d-41d5-bf85-a57642501bd3-delete_volume'" released by "delete_volume" :: held 8.430s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:11.578517 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615-delete_volume'" acquired by "delete_volume" :: waited 0.036s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:11.585130 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=60,cinder:DELETE=2,cinder:UPDATE=31,cinder:INSERT=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:11.725585 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:12.095593 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:21:12.096880 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] Volume volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:21:12.272906 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] deleting rbd volume volume-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:21:12.994452 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2777e460-0431-437b-bc3f-ac22877e26c0 tempest-SnapshotsActionsTest-1795257975 None] Created reservations ['57f5d31d-14ba-432d-908a-c43bae56e0d8', '54321d93-98ab-4738-81f7-88f4ef4614d4', '1b646ab5-33ea-4365-ba3b-1e4968de6e49', '5e9d1364-64c9-463b-9e4d-0c7e7de55f81'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:13.272920 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2777e460-0431-437b-bc3f-ac22877e26c0 tempest-SnapshotsActionsTest-1795257975 None] Delete snapshot completed successfully. May 28 21:21:13.290282 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2777e460-0431-437b-bc3f-ac22877e26c0 tempest-SnapshotsActionsTest-1795257975 None] Lock "b'cinder-71d8b4ce-6fa6-40fd-8722-b2f93b50039a-delete_snapshot'" released by "delete_snapshot" :: held 4.754s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:14.091177 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] Lock "b'cinder-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:14.216999 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:14.449512 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:21:14.450171 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] Volume volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:21:14.616023 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] deleting rbd volume volume-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:21:17.559406 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-536a48a9-4d3c-48e7-abf7-512162477311 tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc'" acquired by "attach_volume" :: waited 0.016s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:17.603647 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c7b91485-83ae-4795-b6e0-bda82b8a4c68 req-9cce81d5-8270-4fdc-a09d-e0819a098dcd tempest-AttachVolumeShelveTestJSON-1592071011 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:21:17.845702 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-536a48a9-4d3c-48e7-abf7-512162477311 tempest-VolumesActionsTest-514564619 None] Attaching volume 367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc to instance aa4578da-78a1-462f-a859-07686f886f99 at mountpoint /dev/sdb on host None. May 28 21:21:18.000399 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-536a48a9-4d3c-48e7-abf7-512162477311 tempest-VolumesActionsTest-514564619 None] Attach volume completed successfully. May 28 21:21:18.006605 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-536a48a9-4d3c-48e7-abf7-512162477311 tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc'" released by "attach_volume" :: held 0.447s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:18.190073 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=68,cinder:UPDATE=24,cinder:INSERT=7,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:18.306625 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] Created reservations ['c0211f87-c9c1-49df-a710-831a9750e9c5', '67b6782e-a8de-43f4-961b-4504e694a53c', 'ae665072-9666-4850-8d5b-ff54179f1755', '2268b82f-e4cc-422c-a4ed-9fffc60ffd50'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:18.392110 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e0194d0f-0244-4803-8710-47178e038898 tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-detach_volume'" acquired by "detach_volume" :: waited 0.016s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:18.392984 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e0194d0f-0244-4803-8710-47178e038898 tempest-VolumesActionsTest-514564619 None] Detaching volume 367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc from instance aa4578da-78a1-462f-a859-07686f886f99. May 28 21:21:18.610459 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:18.668031 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:18.920200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:21:18.920200 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-c7b91485-83ae-4795-b6e0-bda82b8a4c68 req-9cce81d5-8270-4fdc-a09d-e0819a098dcd tempest-AttachVolumeShelveTestJSON-1592071011 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.314s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:21:18.920200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-c7b91485-83ae-4795-b6e0-bda82b8a4c68 req-9cce81d5-8270-4fdc-a09d-e0819a098dcd tempest-AttachVolumeShelveTestJSON-1592071011 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-aca16bed-3410-4df8-b98d-9a6854483c48', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'aca16bed-3410-4df8-b98d-9a6854483c48', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:21:18.923474 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] Deleted volume successfully. May 28 21:21:18.938106 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e0194d0f-0244-4803-8710-47178e038898 tempest-VolumesActionsTest-514564619 None] Detach volume completed successfully. May 28 21:21:18.945532 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e0194d0f-0244-4803-8710-47178e038898 tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-detach_volume'" released by "detach_volume" :: held 0.553s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:19.008607 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f111cae9-407d-4688-9d08-297e78aa490d tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-1896dbc5-1bc4-4ba7-a09c-f8a5a3e84615-delete_volume'" released by "delete_volume" :: held 7.424s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:19.055109 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c7b91485-83ae-4795-b6e0-bda82b8a4c68 req-9cce81d5-8270-4fdc-a09d-e0819a098dcd tempest-AttachVolumeShelveTestJSON-1592071011 None] Connection info returned from driver {'name': 'volumes/volume-aca16bed-3410-4df8-b98d-9a6854483c48', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'aca16bed-3410-4df8-b98d-9a6854483c48', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '0a6d3e3d-a95f-4fec-b76c-e00205ebd570'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:21:19.239770 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c7b91485-83ae-4795-b6e0-bda82b8a4c68 req-9cce81d5-8270-4fdc-a09d-e0819a098dcd tempest-AttachVolumeShelveTestJSON-1592071011 None] attachment_update completed successfully. May 28 21:21:19.516281 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-f988fe9b-43f7-4336-8526-979ab3142238-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:19.676306 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:19.923756 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:21:20.325410 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] deleting rbd volume volume-f988fe9b-43f7-4336-8526-979ab3142238 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:21:21.118592 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=68,cinder:UPDATE=26,cinder:INSERT=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:21.296888 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] Created reservations ['7e0fb935-aa9b-4d91-aa1b-125b0b7f456f', 'ddb17b2f-3c66-4849-b0e7-b437fd52e69b', '6603b0d8-3ba1-424d-931a-63ec05856664', '93022aad-b18a-4549-b2ee-45e8ad4ddb8b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:21.621738 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:21.721183 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:21.867066 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:21:21.872743 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] Deleted volume successfully. May 28 21:21:21.962882 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8870f0a8-619e-42d5-bfad-89e1f3ee3de5 tempest-SnapshotsActionsTest-1795257975 None] Lock "b'cinder-a9fba3fa-3d77-4070-8b79-8adf9ed6bedd-delete_volume'" released by "delete_volume" :: held 7.872s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:27.775407 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:21:27.924731 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] deleting parent snapshot volume-f988fe9b-43f7-4336-8526-979ab3142238.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:21:30.836168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] Created reservations ['f41a01f8-ee21-431e-b2c3-e980b706effb', 'd4614b12-3210-49e7-b019-7a68cbdefed5', '2703bc38-1aca-4307-a2a3-ffbe6d509579', '2e086029-a1e2-47e4-9e6c-529a6c2ccdd9'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:31.893461 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:32.062663 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:32.154173 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:21:32.159074 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] Deleted volume successfully. May 28 21:21:32.270555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2a942488-cc85-4bfa-bfb3-0d3bbccc1a4c tempest-TaggedBootDevicesTest-427054523 None] Lock "b'cinder-f988fe9b-43f7-4336-8526-979ab3142238-delete_volume'" released by "delete_volume" :: held 12.755s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:45.255490 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:INSERT=1,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:46.453225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-bc30591d-33ff-4a68-a8fd-dfcf1bb42edb tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc'" acquired by "attach_volume" :: waited 0.027s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:46.701573 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-bc30591d-33ff-4a68-a8fd-dfcf1bb42edb tempest-VolumesActionsTest-514564619 None] Attaching volume 367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc to instance 717c375c-142d-4538-ada9-ac7ae5e58dbc at mountpoint /dev/sdb on host None. May 28 21:21:46.878064 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-bc30591d-33ff-4a68-a8fd-dfcf1bb42edb tempest-VolumesActionsTest-514564619 None] Attach volume completed successfully. May 28 21:21:46.884629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-bc30591d-33ff-4a68-a8fd-dfcf1bb42edb tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc'" released by "attach_volume" :: held 0.431s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:47.230454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-177a67df-f15c-4822-95e8-659fccad4dff tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-detach_volume'" acquired by "detach_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:47.236561 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-177a67df-f15c-4822-95e8-659fccad4dff tempest-VolumesActionsTest-514564619 None] Detaching volume 367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc from instance 717c375c-142d-4538-ada9-ac7ae5e58dbc. May 28 21:21:47.538084 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-177a67df-f15c-4822-95e8-659fccad4dff tempest-VolumesActionsTest-514564619 None] Detach volume completed successfully. May 28 21:21:47.546786 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-177a67df-f15c-4822-95e8-659fccad4dff tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-detach_volume'" released by "detach_volume" :: held 0.316s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:21:48.796637 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=56,cinder:INSERT=9,cinder:UPDATE=34,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:21:48.878088 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:21:48.901002 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Flow 'volume_create_manager' (ef1bc5b1-faa7-412f-a386-db51f90abb2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:21:48.908154 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ce31ebef-5269-46f5-8f6c-61410b6aa066) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:21:48.927055 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ce31ebef-5269-46f5-8f6c-61410b6aa066) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:21:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesMetadataTest-Volume-470319363',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=07186941-8d82-4daa-a452-a5a5cf78d14b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='1a1fd2902ed9488cb220da0fba115078',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:21:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:21:49Z,use_quota=True,user_id='0d7218edd07846c2b99e993bfebebb74',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:21:48.932686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a6e9a98a-768d-42e5-8a1f-9e01fec93284) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:21:48.937811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a6e9a98a-768d-42e5-8a1f-9e01fec93284) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:21:48.943223 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d1454b0b-df93-417e-8276-f1999dec6ce7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:21:48.947497 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d1454b0b-df93-417e-8276-f1999dec6ce7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '07186941-8d82-4daa-a452-a5a5cf78d14b', 'volume_name': 'volume-07186941-8d82-4daa-a452-a5a5cf78d14b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:21:48.953913 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4994c823-a222-4b9f-bd30-780b3e34d247) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:21:48.958331 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4994c823-a222-4b9f-bd30-780b3e34d247) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:21:48.963398 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dacff56f-df89-42a4-838c-e87db71fa36e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:21:48.964641 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Volume 07186941-8d82-4daa-a452-a5a5cf78d14b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-07186941-8d82-4daa-a452-a5a5cf78d14b', 'volume_size': 1} May 28 21:21:48.965252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] creating volume 'volume-07186941-8d82-4daa-a452-a5a5cf78d14b' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:21:48.966283 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:49.965176 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:21:49.966599 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:21:49.971426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dacff56f-df89-42a4-838c-e87db71fa36e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-07186941-8d82-4daa-a452-a5a5cf78d14b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:21:49.979388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6f042efa-e121-4465-bc09-468c2e862a98) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:21:50.093431 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Volume volume-07186941-8d82-4daa-a452-a5a5cf78d14b (07186941-8d82-4daa-a452-a5a5cf78d14b): created successfully May 28 21:21:50.096872 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6f042efa-e121-4465-bc09-468c2e862a98) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:21:50.137005 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Flow 'volume_create_manager' (ef1bc5b1-faa7-412f-a386-db51f90abb2a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:21:50.220209 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-89ff9cd2-d7be-47a4-8f8a-1e0eace64fa3 tempest-VolumesMetadataTest-1024912381 None] Created volume successfully. May 28 21:21:52.534596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] Lock "b'cinder-07186941-8d82-4daa-a452-a5a5cf78d14b-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:21:52.762649 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:53.020894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:21:53.022196 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] Volume volume-07186941-8d82-4daa-a452-a5a5cf78d14b is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:21:53.152129 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] deleting rbd volume volume-07186941-8d82-4daa-a452-a5a5cf78d14b {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:21:54.087785 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-d76f833b-2345-44bf-b119-314b3391ccd3 tempest-VolumesActionsTest-514564619 None] Running cmd (subprocess): rbd export --pool volumes volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc /opt/stack/data/cinder/conversion/volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-c4b6dd7e-4439-4747-815d-1a66dc2bbd5b --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:21:55.234785 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-d76f833b-2345-44bf-b119-314b3391ccd3 tempest-VolumesActionsTest-514564619 None] CMD "rbd export --pool volumes volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc /opt/stack/data/cinder/conversion/volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-c4b6dd7e-4439-4747-815d-1a66dc2bbd5b --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.145s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:21:55.275461 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.image_utils [None req-d76f833b-2345-44bf-b119-314b3391ccd3 tempest-VolumesActionsTest-514564619 None] c4b6dd7e-4439-4747-815d-1a66dc2bbd5b was raw, no need to convert to raw {{(pid=135921) upload_volume /opt/stack/cinder/cinder/image/image_utils.py:815}} May 28 21:21:58.289767 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] Created reservations ['d76bba4a-b06b-4b3f-be40-4d1a5fc01377', '37501179-85ef-4aca-8e63-5c251345ca9b', '2603b92b-ae41-4a26-948d-d38aa380c746', '4ed9f9ba-05f4-4952-afc1-754c9a76fb4c'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:21:58.587929 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:58.663559 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:21:58.743765 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:21:58.749629 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] Deleted volume successfully. May 28 21:21:58.833290 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-38b345a6-c4b3-40c2-b4b0-2a092ca1069f tempest-VolumesMetadataTest-1024912381 None] Lock "b'cinder-07186941-8d82-4daa-a452-a5a5cf78d14b-delete_volume'" released by "delete_volume" :: held 6.299s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:00.796571 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:22:00.797339 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:00.876979 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:01.038929 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:22:06.785121 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:22:06.826934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Flow 'volume_create_manager' (afebb095-b0ef-4ca8-b096-c158b9cc3211) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:22:06.837629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d0b68b5e-00e1-42a7-90fd-7cfcfa9236dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:06.859979 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d0b68b5e-00e1-42a7-90fd-7cfcfa9236dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:22:06Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneNegativeTest-Volume-1244725990',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=920da980-f5e9-480e-95e7-9f19cc2b04a5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='32ea7778e193493fac70cb7da7c7ba5d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:22:07Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:22:07Z,use_quota=True,user_id='abc82744eb774701b36fcc350a50e509',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:06.866675 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f74e2121-d0bd-4000-a93c-94eb68714641) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:06.872838 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f74e2121-d0bd-4000-a93c-94eb68714641) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:06.879112 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f95a21ad-d0a9-4e8e-8982-126618813689) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:06.887250 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f95a21ad-d0a9-4e8e-8982-126618813689) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '920da980-f5e9-480e-95e7-9f19cc2b04a5', 'volume_name': 'volume-920da980-f5e9-480e-95e7-9f19cc2b04a5', 'volume_size': 2}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:06.895889 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6df62a82-0078-40fe-95a1-4eb372dc8c40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:06.900442 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6df62a82-0078-40fe-95a1-4eb372dc8c40) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:06.907436 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3e945f87-07f9-4c8c-a5db-e870aff77f59) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:06.909302 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Volume 920da980-f5e9-480e-95e7-9f19cc2b04a5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-920da980-f5e9-480e-95e7-9f19cc2b04a5', 'volume_size': 2} May 28 21:22:06.909766 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] creating volume 'volume-920da980-f5e9-480e-95e7-9f19cc2b04a5' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:22:06.910638 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:07.635486 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:22:07.635919 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:22:07.640173 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3e945f87-07f9-4c8c-a5db-e870aff77f59) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-920da980-f5e9-480e-95e7-9f19cc2b04a5', 'volume_size': 2}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:07.644792 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (16a1114f-1e0b-475f-abaf-e39800af0bc1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:07.767722 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Volume volume-920da980-f5e9-480e-95e7-9f19cc2b04a5 (920da980-f5e9-480e-95e7-9f19cc2b04a5): created successfully May 28 21:22:07.771510 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (16a1114f-1e0b-475f-abaf-e39800af0bc1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:07.780743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Flow 'volume_create_manager' (afebb095-b0ef-4ca8-b096-c158b9cc3211) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:22:07.899109 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-61c9c60a-1577-4792-b9bc-3aa776edcaca tempest-VolumesCloneNegativeTest-1915477110 None] Created volume successfully. May 28 21:22:09.070070 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] Lock "b'cinder-920da980-f5e9-480e-95e7-9f19cc2b04a5-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:22:09.223311 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:09.699324 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:22:09.700033 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] Volume volume-920da980-f5e9-480e-95e7-9f19cc2b04a5 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:22:09.928427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] deleting rbd volume volume-920da980-f5e9-480e-95e7-9f19cc2b04a5 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:22:12.746283 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] Created reservations ['89dbe405-4cb2-4a5c-8739-c90a0100f9c3', 'be215f8d-f036-4a36-8ebd-3c0080844230', '4ab7b4bb-11d4-4999-82e2-6a93d89179ad', '98362e1f-edeb-499c-ad49-7c0ed2ecf4c7'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:22:13.027497 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:13.142214 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:13.329525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:22:13.344300 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] Deleted volume successfully. May 28 21:22:13.491971 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f6337695-8825-403e-bcea-db287a981498 tempest-VolumesCloneNegativeTest-1915477110 None] Lock "b'cinder-920da980-f5e9-480e-95e7-9f19cc2b04a5-delete_volume'" released by "delete_volume" :: held 4.422s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:16.362300 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-8edf0118-b07e-49d1-9829-45445c5ec665 req-ca31c138-bbb8-4226-b943-6b39eba0c991 tempest-TaggedBootDevicesTest_v242-1600206585 None] Terminate volume connection completed successfully. May 28 21:22:16.368324 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-8edf0118-b07e-49d1-9829-45445c5ec665 req-ca31c138-bbb8-4226-b943-6b39eba0c991 tempest-TaggedBootDevicesTest_v242-1600206585 None] Deleting attachment 8214d51c-c188-42b7-a71c-00823290c271. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:22:16.745100 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-8edf0118-b07e-49d1-9829-45445c5ec665 req-11b91f5a-eaae-424b-8b7a-ff9638f2cd90 tempest-TaggedBootDevicesTest_v242-1600206585 None] Terminate volume connection completed successfully. May 28 21:22:16.747639 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-8edf0118-b07e-49d1-9829-45445c5ec665 req-11b91f5a-eaae-424b-8b7a-ff9638f2cd90 tempest-TaggedBootDevicesTest_v242-1600206585 None] Deleting attachment d9d2e024-3f33-45e9-8e34-07d8213df050. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:22:17.074424 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-8edf0118-b07e-49d1-9829-45445c5ec665 req-8bc890ff-a8c3-4fd9-948a-3309b641030c tempest-TaggedBootDevicesTest_v242-1600206585 None] Terminate volume connection completed successfully. May 28 21:22:17.080318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-8edf0118-b07e-49d1-9829-45445c5ec665 req-8bc890ff-a8c3-4fd9-948a-3309b641030c tempest-TaggedBootDevicesTest_v242-1600206585 None] Deleting attachment 0310d57d-44fa-4897-a2a7-e460e5c53cca. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:22:23.495993 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=42,cinder:UPDATE=22,cinder:INSERT=1,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:23.496913 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=51,cinder:UPDATE=19,cinder:DELETE=3,cinder:INSERT=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:23.506014 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-afd5a436-f329-4caf-a019-4b6a6d80d229 req-9fccbdbc-2c18-4648-bc1f-36b97e6789a7 tempest-AttachVolumeShelveTestJSON-1592071011 None] Terminate volume connection completed successfully. May 28 21:22:23.506606 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-afd5a436-f329-4caf-a019-4b6a6d80d229 req-9fccbdbc-2c18-4648-bc1f-36b97e6789a7 tempest-AttachVolumeShelveTestJSON-1592071011 None] Deleting attachment 0a6d3e3d-a95f-4fec-b76c-e00205ebd570. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:22:32.589184 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-0aae4e20-4ac4-4c87-a853-064fb70796fb-delete_volume'" acquired by "delete_volume" :: waited 0.037s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:22:32.822880 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:32.990361 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:22:32.990855 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume volume-0aae4e20-4ac4-4c87-a853-064fb70796fb is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:22:33.081411 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] deleting rbd volume volume-0aae4e20-4ac4-4c87-a853-064fb70796fb {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:22:36.002372 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] Created reservations ['c789faea-ee0f-4a8f-9e24-6255ac5da258', '19364c1f-b747-4995-9548-6fea2df18a39', '808837ff-4721-4757-8859-3cc3c086303f', 'ef25bb09-e992-4123-9c6a-ae1be565b1e0'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:22:36.290684 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:36.371482 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:36.467645 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:22:36.487162 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] Deleted volume successfully. May 28 21:22:36.737980 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d76f833b-2345-44bf-b119-314b3391ccd3 tempest-VolumesActionsTest-514564619 None] Uploaded volume to glance image-id: c4b6dd7e-4439-4747-815d-1a66dc2bbd5b. {{(pid=135921) copy_volume_to_image /opt/stack/cinder/cinder/volume/manager.py:1751}} May 28 21:22:36.758381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-abb5f7c9-1b6d-4fbc-be3d-3c15a9219e2b tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-0aae4e20-4ac4-4c87-a853-064fb70796fb-delete_volume'" released by "delete_volume" :: held 4.169s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:36.847253 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d76f833b-2345-44bf-b119-314b3391ccd3 tempest-VolumesActionsTest-514564619 None] Copy volume to image completed successfully. May 28 21:22:37.325016 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-50730637-3c48-4270-bed2-229c25f4f463-delete_volume'" acquired by "delete_volume" :: waited 0.021s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:22:37.485534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:37.755587 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:22:37.756135 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Volume volume-50730637-3c48-4270-bed2-229c25f4f463 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:22:37.877017 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] deleting rbd volume volume-50730637-3c48-4270-bed2-229c25f4f463 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:22:39.727332 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:22:39.848191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:40.037142 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:22:40.038420 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] Volume volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:22:40.145919 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] deleting rbd volume volume-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:22:41.787024 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Created reservations ['084bacd1-d59c-4442-8a72-a4d10ba351aa', 'f911bb74-7c8a-45e7-8f7e-a99a9d009d42', '9ab989ac-51b9-4778-aa17-75a81ad10d39', '111bda01-7dea-4632-a60f-93a2fc657b6a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:22:42.059269 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:42.132268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:42.206352 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:22:42.221011 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Deleted volume successfully. May 28 21:22:42.339588 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a634d372-b54a-4293-9a74-7036b95d04a1 tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-50730637-3c48-4270-bed2-229c25f4f463-delete_volume'" released by "delete_volume" :: held 5.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:43.030023 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-26cabed5-9ce1-42aa-b829-f68d323905f3-delete_volume'" acquired by "delete_volume" :: waited 0.023s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:22:43.235736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:43.425137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:22:43.497622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] deleting rbd volume volume-26cabed5-9ce1-42aa-b829-f68d323905f3 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:22:43.699316 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] Created reservations ['4ffee4b5-0d06-4a85-9864-f5bdda518c23', '5e538d27-610e-4216-9c94-661087555018', '84f61856-2dfe-4028-be4c-04ddd54a2b61', '6d409a88-c18e-4b8c-909c-11a655e67ded'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:22:44.019718 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:44.163941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:44.260549 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:22:44.266306 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] Deleted volume successfully. May 28 21:22:44.347793 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-516a0f08-11fb-460d-8aa4-d81205d6da1b tempest-VolumesActionsTest-514564619 None] Lock "b'cinder-367dcfbc-6a49-428d-afc6-3b4fc3c1e1fc-delete_volume'" released by "delete_volume" :: held 4.618s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:46.564907 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:22:46.694090 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] deleting parent snapshot volume-26cabed5-9ce1-42aa-b829-f68d323905f3.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:22:48.989942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] Created reservations ['b3eb6867-2c49-4222-90bc-13530ac09639', '60648183-7aa4-4c3f-930e-818016766fde', '1fba3ed3-8388-4b49-be9c-c59ce8b47499', 'd6e9d89e-21d8-4861-baef-9bf64253e87d'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:22:49.115493 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=55,cinder:UPDATE=30,cinder:INSERT=13,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:49.218218 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:49.363832 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:49.439297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:22:49.446340 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] Deleted volume successfully. May 28 21:22:49.451529 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=58,cinder:UPDATE=38,cinder:INSERT=3,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:22:49.560333 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-39bb646f-af4c-4fa3-a270-aab6a0e8a6d5 tempest-TaggedBootDevicesTest_v242-1600206585 None] Lock "b'cinder-26cabed5-9ce1-42aa-b829-f68d323905f3-delete_volume'" released by "delete_volume" :: held 6.532s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:51.675943 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:22:51.745756 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Flow 'volume_create_manager' (48dd1577-f2f3-4376-aacc-8a194582be56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:22:51.777136 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bcee0c12-e33f-4c2e-bcd5-aff5de2811db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:51.816524 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bcee0c12-e33f-4c2e-bcd5-aff5de2811db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:22:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-776034186',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b19f2620-291e-44fe-9d8d-be45cbc3c158,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='753b86e7a74c40369a4697ba5e7920d7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:22:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:22:51Z,use_quota=True,user_id='d8a35655682e4ece8da56d74d6f535cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:51.842730 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5f146c85-34bf-455a-b190-c070b6305594) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:51.854405 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5f146c85-34bf-455a-b190-c070b6305594) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:51.865609 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b357dac7-1305-47db-a3db-49663b2de128) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:52.033803 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b357dac7-1305-47db-a3db-49663b2de128) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'b19f2620-291e-44fe-9d8d-be45cbc3c158', 'volume_name': 'volume-b19f2620-291e-44fe-9d8d-be45cbc3c158', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:52.039589 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aec7cb55-362a-4e56-8995-c57f9da512c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:52.047327 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (aec7cb55-362a-4e56-8995-c57f9da512c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:52.082632 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dec5478b-34bd-47a1-aaf8-d14f2cc9a74b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:52.084002 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Volume b19f2620-291e-44fe-9d8d-be45cbc3c158: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-b19f2620-291e-44fe-9d8d-be45cbc3c158', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:22:52.084768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Cloning b19f2620-291e-44fe-9d8d-be45cbc3c158 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:22:52.098289 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:22:52.196581 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 22, 52, 110732)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:22:52.197189 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:22:52.197655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:22:52.204312 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.106s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:22:52.205495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:22:52.218235 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 22, 52, 212876)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:22:52.222176 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:22:52.223143 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:22:52.332327 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:52.549721 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] creating snapshot='volume-b19f2620-291e-44fe-9d8d-be45cbc3c158.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:22:53.853455 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-b19f2620-291e-44fe-9d8d-be45cbc3c158.clone_snap' to 'volume-b19f2620-291e-44fe-9d8d-be45cbc3c158' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:22:54.794100 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:22:54.795370 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:22:54.795784 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:22:54.966239 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:22:54.967791 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Marking volume b19f2620-291e-44fe-9d8d-be45cbc3c158 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:22:55.052722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to b19f2620-291e-44fe-9d8d-be45cbc3c158. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:22:55.053737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Creating volume glance metadata for volume b19f2620-291e-44fe-9d8d-be45cbc3c158 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:22:55.228828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dec5478b-34bd-47a1-aaf8-d14f2cc9a74b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b19f2620-291e-44fe-9d8d-be45cbc3c158', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:55.235796 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (caf76e3b-ebd9-4699-a02b-7c798aee9f01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:55.315468 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Volume volume-b19f2620-291e-44fe-9d8d-be45cbc3c158 (b19f2620-291e-44fe-9d8d-be45cbc3c158): created successfully May 28 21:22:55.320978 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (caf76e3b-ebd9-4699-a02b-7c798aee9f01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:55.330328 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Flow 'volume_create_manager' (48dd1577-f2f3-4376-aacc-8a194582be56) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:22:55.441494 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-331d7265-a067-4f7c-ac77-860155453266 tempest-VolumesExtendTest-1745263701 None] Created volume successfully. May 28 21:22:56.019229 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f1c75799-b9c2-47a1-82da-782c47d63504 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:56.845404 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f1c75799-b9c2-47a1-82da-782c47d63504 tempest-VolumesExtendTest-1745263701 None] Extend volume from 1 GB to 2 GB. {{(pid=135921) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1886}} May 28 21:22:57.021080 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f1c75799-b9c2-47a1-82da-782c47d63504 tempest-VolumesExtendTest-1745263701 None] Extend volume completed successfully. May 28 21:22:58.494201 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:22:58.545962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Flow 'volume_create_manager' (7f7808e8-bae4-4cea-bba6-b1fcd9e6f0de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:22:58.561026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c819c024-1770-4493-b9cc-ea27bb22d76a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:58.590980 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c819c024-1770-4493-b9cc-ea27bb22d76a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:22:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesExtendTest-Volume-921072607',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=77731526-b397-489b-831a-782ea6c53c4f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='753b86e7a74c40369a4697ba5e7920d7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:22:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:22:58Z,use_quota=True,user_id='d8a35655682e4ece8da56d74d6f535cd',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:58.604278 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (587433f5-f265-41a8-af0d-84d231f8d0b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:58.616081 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (587433f5-f265-41a8-af0d-84d231f8d0b4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:58.628945 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b2337e1-6e7a-48f1-92cb-4225ff8bcff9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:58.638405 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b2337e1-6e7a-48f1-92cb-4225ff8bcff9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '77731526-b397-489b-831a-782ea6c53c4f', 'volume_name': 'volume-77731526-b397-489b-831a-782ea6c53c4f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:58.655802 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (72469b27-2a8f-458f-8a18-6f0acff7c8c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:58.660213 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (72469b27-2a8f-458f-8a18-6f0acff7c8c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:58.665292 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (144dcbe0-8ec9-4097-aec8-b5dbc59574fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:58.666431 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Volume 77731526-b397-489b-831a-782ea6c53c4f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-77731526-b397-489b-831a-782ea6c53c4f', 'volume_size': 1} May 28 21:22:58.666821 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] creating volume 'volume-77731526-b397-489b-831a-782ea6c53c4f' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:22:58.667592 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:22:59.084665 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:22:59.086480 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:22:59.092308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (144dcbe0-8ec9-4097-aec8-b5dbc59574fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-77731526-b397-489b-831a-782ea6c53c4f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:59.100026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (04185a1e-8143-46c4-a871-43d9f903a5b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:22:59.155583 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Volume volume-77731526-b397-489b-831a-782ea6c53c4f (77731526-b397-489b-831a-782ea6c53c4f): created successfully May 28 21:22:59.159951 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (04185a1e-8143-46c4-a871-43d9f903a5b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:22:59.168586 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Flow 'volume_create_manager' (7f7808e8-bae4-4cea-bba6-b1fcd9e6f0de) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:22:59.275346 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-36ff0395-c50b-4874-8a4d-2313d299d8f2 tempest-VolumesExtendTest-1745263701 None] Created volume successfully. May 28 21:23:00.120027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2dd9a3b6-a2a4-48c3-afe9-a7f522aad12c tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:00.793377 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:23:00.794360 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:01.020101 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:01.099207 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:02.389510 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2dd9a3b6-a2a4-48c3-afe9-a7f522aad12c tempest-VolumesExtendTest-1745263701 None] Create snapshot completed successfully May 28 21:23:03.567417 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-724d1f8d-9174-492b-9da1-7434916b76b5 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:04.444595 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-724d1f8d-9174-492b-9da1-7434916b76b5 tempest-VolumesExtendTest-1745263701 None] Extend volume from 1 GB to 2 GB. {{(pid=135921) extend_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1886}} May 28 21:23:04.709793 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-724d1f8d-9174-492b-9da1-7434916b76b5 tempest-VolumesExtendTest-1745263701 None] Extend volume completed successfully. May 28 21:23:06.177470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9172e7f5-af7b-4b81-8a35-e0f3f564ff60 tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-78b194b2-caf5-49ff-a2d5-0b52a2240855-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:06.182090 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9172e7f5-af7b-4b81-8a35-e0f3f564ff60 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:08.373125 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9172e7f5-af7b-4b81-8a35-e0f3f564ff60 tempest-VolumesExtendTest-1745263701 None] Created reservations ['eda8c0c8-0ebc-43f8-bb6b-a4572237077a', 'ae4412ba-9a2a-417c-b9d5-135d41e82c0a', '11fbdf4f-dc10-49a9-8708-b10fd7d2ff22', 'f0afa60e-834e-4477-b5dd-1912cb9f5a40'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:08.692545 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9172e7f5-af7b-4b81-8a35-e0f3f564ff60 tempest-VolumesExtendTest-1745263701 None] Delete snapshot completed successfully. May 28 21:23:08.698658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9172e7f5-af7b-4b81-8a35-e0f3f564ff60 tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-78b194b2-caf5-49ff-a2d5-0b52a2240855-delete_snapshot'" released by "delete_snapshot" :: held 2.521s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:09.625120 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-77731526-b397-489b-831a-782ea6c53c4f-delete_volume'" acquired by "delete_volume" :: waited 0.020s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:09.783771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:09.943909 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:09.944335 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] Volume volume-77731526-b397-489b-831a-782ea6c53c4f is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:23:10.115314 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] deleting rbd volume volume-77731526-b397-489b-831a-782ea6c53c4f {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:23:12.865105 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] Created reservations ['9dc858e6-e4be-41f9-90ef-25e1a978a012', 'cf7bbb42-2724-411f-b909-ac184b79b16b', 'ed3744c0-4d0c-4968-8ca6-b299bb79b4f5', '6c68ee0f-6456-4970-9327-dae0176aee4a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:13.161146 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:13.213308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:13.338465 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:13.347216 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] Deleted volume successfully. May 28 21:23:13.457789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-429625a9-51fa-4cc2-b255-4d9378144456 tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-77731526-b397-489b-831a-782ea6c53c4f-delete_volume'" released by "delete_volume" :: held 3.833s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:14.353990 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-b19f2620-291e-44fe-9d8d-be45cbc3c158-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:14.395354 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:DELETE=1,cinder:SELECT=61,cinder:UPDATE=28,cinder:INSERT=10 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:14.556758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:14.766305 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:14.882833 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] deleting rbd volume volume-b19f2620-291e-44fe-9d8d-be45cbc3c158 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:23:18.007449 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:23:18.128102 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] deleting parent snapshot volume-b19f2620-291e-44fe-9d8d-be45cbc3c158.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:23:20.773488 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=62,cinder:INSERT=10,cinder:UPDATE=24,cinder:DELETE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:20.881186 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] Created reservations ['c5ca2942-e360-43a1-a061-67259320fff0', '4030eed6-3cfc-4235-8951-f42686771685', 'fb810683-ed38-4baf-904b-a2cb14724e30', '8bdb9a50-5211-4e83-bb9a-0ed27f1edca1'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:21.166641 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:21.295564 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:21.366074 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:21.371534 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] Deleted volume successfully. May 28 21:23:21.516435 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d90ed586-1934-468a-be36-b366fc933beb tempest-VolumesExtendTest-1745263701 None] Lock "b'cinder-b19f2620-291e-44fe-9d8d-be45cbc3c158-delete_volume'" released by "delete_volume" :: held 7.162s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:24.246923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:23:24.359099 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Flow 'volume_create_manager' (f98ced16-a48a-46e4-809d-6bc53a8098d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:24.378548 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7e186bd8-1790-46b4-97ba-472b52e7b096) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:24.418805 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7e186bd8-1790-46b4-97ba-472b52e7b096) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-180861509',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e03888d6-271e-4e81-852c-f74bf4bd491e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ce4ccb9f2fb945e19bfed037a38ea59d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:23:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:23:24Z,use_quota=True,user_id='f525c29419db48e684dd53c758abb74b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:24.435225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4a793887-ee6d-49fd-a8cb-6ff6ba95184a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:24.439806 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4a793887-ee6d-49fd-a8cb-6ff6ba95184a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:24.453340 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0ae096d6-305d-44b4-a75b-3f58f6ef7e8c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:24.470833 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0ae096d6-305d-44b4-a75b-3f58f6ef7e8c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e03888d6-271e-4e81-852c-f74bf4bd491e', 'volume_name': 'volume-e03888d6-271e-4e81-852c-f74bf4bd491e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:24.476278 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2318353c-c27c-41f7-aebc-c45e2b614454) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:24.487728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2318353c-c27c-41f7-aebc-c45e2b614454) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:24.509633 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c32569f7-2c19-442c-8e32-8cd16f01e52b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:24.510964 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume e03888d6-271e-4e81-852c-f74bf4bd491e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e03888d6-271e-4e81-852c-f74bf4bd491e', 'volume_size': 1} May 28 21:23:24.518376 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] creating volume 'volume-e03888d6-271e-4e81-852c-f74bf4bd491e' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:23:24.520178 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:24.940823 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:23:24.947191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:23:24.947191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c32569f7-2c19-442c-8e32-8cd16f01e52b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e03888d6-271e-4e81-852c-f74bf4bd491e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:24.963974 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (690a64f8-56c9-4513-bd37-ad1fbefc6f8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:25.046620 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume volume-e03888d6-271e-4e81-852c-f74bf4bd491e (e03888d6-271e-4e81-852c-f74bf4bd491e): created successfully May 28 21:23:25.050212 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (690a64f8-56c9-4513-bd37-ad1fbefc6f8a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:25.056573 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Flow 'volume_create_manager' (f98ced16-a48a-46e4-809d-6bc53a8098d6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:25.159736 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5647b516-e7cf-4119-aa5b-6f1c87b6e77e tempest-VolumeQuotasAdminTestJSON-1835509782 None] Created volume successfully. May 28 21:23:26.898470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Lock "b'cinder-e03888d6-271e-4e81-852c-f74bf4bd491e-delete_volume'" acquired by "delete_volume" :: waited 0.038s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:27.054495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:27.377536 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:27.378268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume volume-e03888d6-271e-4e81-852c-f74bf4bd491e is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:23:27.538090 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] deleting rbd volume volume-e03888d6-271e-4e81-852c-f74bf4bd491e {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:23:30.852426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Created reservations ['989d6280-e96d-434d-87fd-db650813b2e5', '38b762af-191d-4d85-8b36-cbb5249b5f84', 'bc99b8ba-6bae-43e0-8309-142f5615567d', 'a856e0a7-06c7-46b3-a94b-d8d0e221bc17'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:31.172701 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:31.324481 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:31.489614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:31.497284 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Deleted volume successfully. May 28 21:23:31.722736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-341c4198-c68c-4646-8a48-89ca51ebd033 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Lock "b'cinder-e03888d6-271e-4e81-852c-f74bf4bd491e-delete_volume'" released by "delete_volume" :: held 4.815s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:33.213547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:23:33.243992 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Flow 'volume_create_manager' (2aa3bfd6-e9ab-4217-b415-e3def02dc869) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:33.251117 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a885a7fe-00f5-4ee0-8bbc-6b0bbc800ba4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.275914 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a885a7fe-00f5-4ee0-8bbc-6b0bbc800ba4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasAdminTestJSON-Volume-536796810',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='ce4ccb9f2fb945e19bfed037a38ea59d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:23:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:23:33Z,use_quota=True,user_id='f525c29419db48e684dd53c758abb74b',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.281228 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b3582b22-459f-42f1-946d-c37c473cd03d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.285789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b3582b22-459f-42f1-946d-c37c473cd03d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.290927 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6590956c-3432-4255-b1f2-f9234dba7ec5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.295633 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6590956c-3432-4255-b1f2-f9234dba7ec5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4', 'volume_name': 'volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.300746 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (792f9f77-f21a-4eec-95ba-4b4c6f17d53a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.305002 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (792f9f77-f21a-4eec-95ba-4b4c6f17d53a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.309676 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0ee15687-7c52-436f-8ca0-90ecd63eabbe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.310989 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4', 'volume_size': 1} May 28 21:23:33.311585 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] creating volume 'volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:23:33.312347 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:33.430234 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:23:33.452427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Flow 'volume_create_manager' (3c3fa563-b1c9-43f5-b5b7-4234b4d32843) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:33.459611 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b4eb7d31-8840-4498-891d-459a110cdc4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.478783 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b4eb7d31-8840-4498-891d-459a110cdc4a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-931179225',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cf706296-6b08-411c-aad7-c87d118555dd,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='e0352bb38c2c4a87bed4a2ca8dbbf437',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:23:33Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:23:33Z,use_quota=True,user_id='15841f513249448081439041b58d091c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.484103 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18adedff-ac92-4aca-91ec-92a9a1279293) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.488999 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (18adedff-ac92-4aca-91ec-92a9a1279293) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.505758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (48d03b62-d24d-470e-9a6c-5d6063376d47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.505758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (48d03b62-d24d-470e-9a6c-5d6063376d47) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cf706296-6b08-411c-aad7-c87d118555dd', 'volume_name': 'volume-cf706296-6b08-411c-aad7-c87d118555dd', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.505758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59e7f063-d1d6-4ba8-8a29-ffb5bafa1c14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.516588 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59e7f063-d1d6-4ba8-8a29-ffb5bafa1c14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.529076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5c6dc76c-1b24-4e2b-b082-4987835d9103) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.531544 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Volume cf706296-6b08-411c-aad7-c87d118555dd: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cf706296-6b08-411c-aad7-c87d118555dd', 'volume_size': 1} May 28 21:23:33.531544 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] creating volume 'volume-cf706296-6b08-411c-aad7-c87d118555dd' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:23:33.533304 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:33.887767 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:23:33.889191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:23:33.892063 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0ee15687-7c52-436f-8ca0-90ecd63eabbe) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:33.898328 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (eb6bfade-3d4e-4975-8b55-c6fd0f3337c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:33.996610 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Volume volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4 (a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4): created successfully May 28 21:23:34.000419 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (eb6bfade-3d4e-4975-8b55-c6fd0f3337c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:34.006466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Flow 'volume_create_manager' (2aa3bfd6-e9ab-4217-b415-e3def02dc869) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:34.092019 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0b957487-1630-4042-8ecb-b920e34e62f8 tempest-VolumeQuotasAdminTestJSON-1835509782 None] Created volume successfully. May 28 21:23:34.238815 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:23:34.240697 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:23:34.249378 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5c6dc76c-1b24-4e2b-b082-4987835d9103) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cf706296-6b08-411c-aad7-c87d118555dd', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:34.262354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (98e1f726-2b21-45f0-841b-918426b4a193) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:34.373288 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Volume volume-cf706296-6b08-411c-aad7-c87d118555dd (cf706296-6b08-411c-aad7-c87d118555dd): created successfully May 28 21:23:34.380079 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (98e1f726-2b21-45f0-841b-918426b4a193) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:34.387972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Flow 'volume_create_manager' (3c3fa563-b1c9-43f5-b5b7-4234b4d32843) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:34.459846 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e3bbe01f-3c96-4cc5-97d9-5a367f14e2f3 tempest-VolumesListTestJSON-1352519245 None] Created volume successfully. May 28 21:23:36.326245 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:23:36.359822 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Flow 'volume_create_manager' (d74755d8-99c8-40ee-980d-c6e143567771) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:36.366843 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fa6e13d2-cfda-4fd8-8515-f316704a76ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:36.383894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fa6e13d2-cfda-4fd8-8515-f316704a76ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-921483716',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9bd98859-31ef-4131-9251-5eeea971c400,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='e0352bb38c2c4a87bed4a2ca8dbbf437',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:23:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:23:36Z,use_quota=True,user_id='15841f513249448081439041b58d091c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:36.405381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb156081-0200-46e1-88c9-de3df002b91e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:36.405381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cb156081-0200-46e1-88c9-de3df002b91e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:36.405381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c0dc958d-a245-44e8-bada-7b0b7d0c952c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:36.405911 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c0dc958d-a245-44e8-bada-7b0b7d0c952c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9bd98859-31ef-4131-9251-5eeea971c400', 'volume_name': 'volume-9bd98859-31ef-4131-9251-5eeea971c400', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:36.409596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (16b6590c-597c-447e-8737-2abdb3d3bbda) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:36.413392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (16b6590c-597c-447e-8737-2abdb3d3bbda) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:36.418079 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (af9f22e8-9207-4569-9031-937414087762) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:36.419141 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Volume 9bd98859-31ef-4131-9251-5eeea971c400: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9bd98859-31ef-4131-9251-5eeea971c400', 'volume_size': 1} May 28 21:23:36.419530 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] creating volume 'volume-9bd98859-31ef-4131-9251-5eeea971c400' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:23:36.420172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:36.885032 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b9734cd5-9821-4366-8bc8-057fa6aa7457 tempest-VolumeQuotasAdminTestJSON-1848902208 None] Transfer volume completed successfully. May 28 21:23:36.987547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:23:36.989211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:23:37.005915 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (af9f22e8-9207-4569-9031-937414087762) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9bd98859-31ef-4131-9251-5eeea971c400', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:37.021790 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d4130ebd-fd02-4757-9bf8-a13d899946cf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:37.131278 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Volume volume-9bd98859-31ef-4131-9251-5eeea971c400 (9bd98859-31ef-4131-9251-5eeea971c400): created successfully May 28 21:23:37.135174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d4130ebd-fd02-4757-9bf8-a13d899946cf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:37.143120 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Flow 'volume_create_manager' (d74755d8-99c8-40ee-980d-c6e143567771) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:37.303358 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-905073a3-9095-44a2-9b66-76de1b604f9d tempest-VolumesListTestJSON-1352519245 None] Created volume successfully. May 28 21:23:38.361732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] Lock "b'cinder-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:38.498076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:38.640200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:38.640678 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] Volume volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:23:38.697965 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] deleting rbd volume volume-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:23:39.236323 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:23:39.256318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Flow 'volume_create_manager' (e31019df-c896-4096-bcc3-22456cab2fce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:39.263259 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9c3b4a39-9031-471b-9a86-f6540fcfa37c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:39.280688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9c3b4a39-9031-471b-9a86-f6540fcfa37c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:38Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListTestJSON-Volume-298286401',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=06c30661-edb2-4789-9428-54931ae1c2ca,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='e0352bb38c2c4a87bed4a2ca8dbbf437',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:23:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:23:39Z,use_quota=True,user_id='15841f513249448081439041b58d091c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:39.286840 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c20ab23f-78f0-44ad-833a-07b26bab8363) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:39.291275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c20ab23f-78f0-44ad-833a-07b26bab8363) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:39.297323 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3beb5b4-9f59-41e9-9717-7d8b56fdef7d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:39.301929 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b3beb5b4-9f59-41e9-9717-7d8b56fdef7d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '06c30661-edb2-4789-9428-54931ae1c2ca', 'volume_name': 'volume-06c30661-edb2-4789-9428-54931ae1c2ca', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:39.307063 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4dd656bc-87ea-4d44-806d-109b2b456f1d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:39.311661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4dd656bc-87ea-4d44-806d-109b2b456f1d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:39.316657 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8bc1ba03-df0b-4ee3-a8c4-0c2a73263718) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:39.317767 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Volume 06c30661-edb2-4789-9428-54931ae1c2ca: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-06c30661-edb2-4789-9428-54931ae1c2ca', 'volume_size': 1} May 28 21:23:39.318190 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] creating volume 'volume-06c30661-edb2-4789-9428-54931ae1c2ca' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:23:39.319185 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:40.175922 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:23:40.181780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:23:40.181780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8bc1ba03-df0b-4ee3-a8c4-0c2a73263718) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-06c30661-edb2-4789-9428-54931ae1c2ca', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:40.187824 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2d10f008-f799-40f5-8e4c-d92533a73539) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:40.313326 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Volume volume-06c30661-edb2-4789-9428-54931ae1c2ca (06c30661-edb2-4789-9428-54931ae1c2ca): created successfully May 28 21:23:40.317515 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2d10f008-f799-40f5-8e4c-d92533a73539) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:40.342673 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Flow 'volume_create_manager' (e31019df-c896-4096-bcc3-22456cab2fce) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:40.441517 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e2b274be-1c89-49f8-b736-3b2b240b510f tempest-VolumesListTestJSON-1352519245 None] Created volume successfully. May 28 21:23:41.786953 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=55,cinder:INSERT=5,cinder:UPDATE=35,cinder:DELETE=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:41.897286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] Created reservations ['c09cb03a-0878-49b6-a75d-a53e8420fb37', '5e8d4e16-cafc-47f7-b3fa-dd43a40a9e2f', '7cce159e-d547-4238-a235-dd30b9690dd5', 'e6708df2-6e86-4aa9-a30f-b6b090e738fc'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:42.223974 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:42.308523 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:42.431769 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:42.439354 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] Deleted volume successfully. May 28 21:23:42.541573 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-986b8020-0e45-4524-9b7d-677f4222a7ae tempest-VolumeQuotasAdminTestJSON-327717415 None] Lock "b'cinder-a1ec20e0-4d4c-4ed3-ba98-fdf5157f21e4-delete_volume'" released by "delete_volume" :: held 4.180s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:42.979699 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:23:43.030230 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Flow 'volume_create_manager' (fadb494a-3cfb-4df4-af4b-d6a8e676cb22) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:43.044517 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4d47e286-99a5-4ba5-b190-6a17673a8f79) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:43.082957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4d47e286-99a5-4ba5-b190-6a17673a8f79) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:42Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-970535380',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bd319c3c-84f3-4f60-b04f-30c14dbe419f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5a8841fa441d413a92a15510a9d082c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:23:43Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:23:43Z,use_quota=True,user_id='857a887b6c7c41588afb2845eed6f309',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:43.090105 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6a7b3feb-f22d-48b5-a57f-16fea00bc223) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:43.095889 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6a7b3feb-f22d-48b5-a57f-16fea00bc223) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:43.102059 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f0bbad73-2482-4fac-9806-052b7fcea7c4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:43.115465 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f0bbad73-2482-4fac-9806-052b7fcea7c4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bd319c3c-84f3-4f60-b04f-30c14dbe419f', 'volume_name': 'volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:43.121926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6c8b377-3a47-4623-886b-713241c2b4a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:43.126913 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6c8b377-3a47-4623-886b-713241c2b4a1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:43.132670 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (61e6d9ba-d50b-466f-8ea3-b2e7ecd7b16c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:43.134778 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume bd319c3c-84f3-4f60-b04f-30c14dbe419f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f', 'volume_size': 1} May 28 21:23:43.135555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] creating volume 'volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:23:43.137427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:43.603620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:23:43.604120 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:23:43.608364 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (61e6d9ba-d50b-466f-8ea3-b2e7ecd7b16c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:43.623266 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3e84191f-7df3-40c5-b656-dac489098b87) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:23:43.696666 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f (bd319c3c-84f3-4f60-b04f-30c14dbe419f): created successfully May 28 21:23:43.701387 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3e84191f-7df3-40c5-b656-dac489098b87) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:23:43.714418 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Flow 'volume_create_manager' (fadb494a-3cfb-4df4-af4b-d6a8e676cb22) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:23:43.839736 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d8283091-1165-48b4-932d-11bc0279ebe4 tempest-AttachVolumeShelveTestJSON-1592071011 None] Created volume successfully. May 28 21:23:43.845651 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=58,cinder:UPDATE=32,cinder:INSERT=7,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:23:47.125499 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] Lock "b'cinder-06c30661-edb2-4789-9428-54931ae1c2ca-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:47.231391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:47.513310 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:47.521241 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] Volume volume-06c30661-edb2-4789-9428-54931ae1c2ca is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:23:47.603270 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] deleting rbd volume volume-06c30661-edb2-4789-9428-54931ae1c2ca {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:23:50.994036 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] Created reservations ['b186f5de-0f2a-4d4b-965d-ae50095b848b', 'd4006ab6-7736-41a4-824c-5e7223d5c7d6', '5360dc24-3212-4256-a4c2-54f92986c196', '96d25979-20a1-4247-81ed-f3757f0c2856'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:51.240722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:51.440823 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:51.590416 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:51.609343 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] Deleted volume successfully. May 28 21:23:51.747718 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e98bcffd-aabf-446b-852c-7783d3f07281 tempest-VolumesListTestJSON-1352519245 None] Lock "b'cinder-06c30661-edb2-4789-9428-54931ae1c2ca-delete_volume'" released by "delete_volume" :: held 4.622s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:51.953080 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] Lock "b'cinder-9bd98859-31ef-4131-9251-5eeea971c400-delete_volume'" acquired by "delete_volume" :: waited 0.017s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:52.161782 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:52.337381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:52.338003 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] Volume volume-9bd98859-31ef-4131-9251-5eeea971c400 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:23:52.463592 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] deleting rbd volume volume-9bd98859-31ef-4131-9251-5eeea971c400 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:23:55.943741 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] Created reservations ['1dbf207d-4272-4779-88f5-2fd4dafb4d9f', '5c71b406-34a1-4cbd-aa6b-b468c301e0fc', '70cccf94-e5ea-4750-bde1-1fffa828a84b', 'b9b68218-918f-47a2-b65e-e346c0e96cd3'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:23:56.286213 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:56.389525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:56.541749 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:23:56.548148 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] Deleted volume successfully. May 28 21:23:56.671195 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-de682f25-5172-429e-9eee-be15c40bafe2 tempest-VolumesListTestJSON-1352519245 None] Lock "b'cinder-9bd98859-31ef-4131-9251-5eeea971c400-delete_volume'" released by "delete_volume" :: held 4.719s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:23:56.872267 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] Lock "b'cinder-cf706296-6b08-411c-aad7-c87d118555dd-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:23:57.032602 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:23:57.227687 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:23:57.228438 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] Volume volume-cf706296-6b08-411c-aad7-c87d118555dd is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:23:57.303190 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] deleting rbd volume volume-cf706296-6b08-411c-aad7-c87d118555dd {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:24:00.230586 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:00.254167 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Flow 'volume_create_manager' (f524795a-ce3f-494b-8b28-f4eb37e80709) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:00.261635 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1c0f53c-6ca6-4a35-9269-0567e61a6bb8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:00.293305 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d1c0f53c-6ca6-4a35-9269-0567e61a6bb8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:23:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesAccessTest-Volume-585934817',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3c555a25-8b45-41a8-8c01-5232dc99c831,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2169bc0792a74919bbe07258b5ae5cc4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:00Z,use_quota=True,user_id='a81fdaea686041ac96619055a9371731',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(430d5002-3a75-4a22-9a9e-cc7d61983ed7),volume_type_id=430d5002-3a75-4a22-9a9e-cc7d61983ed7)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:00.301923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a523d0fd-e1d6-4e7a-8537-43df56dca69a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:00.308500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a523d0fd-e1d6-4e7a-8537-43df56dca69a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:00.320155 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6889dc7c-f659-4cdb-877f-3964606c7801) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:00.326128 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6889dc7c-f659-4cdb-877f-3964606c7801) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3c555a25-8b45-41a8-8c01-5232dc99c831', 'volume_name': 'volume-3c555a25-8b45-41a8-8c01-5232dc99c831', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:00.332280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (08a04971-f56f-41a0-b8eb-e89d8df40d05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:00.337873 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (08a04971-f56f-41a0-b8eb-e89d8df40d05) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:00.343797 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f6cc7722-9865-4aa2-acbd-7766cf06bd71) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:00.345543 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Volume 3c555a25-8b45-41a8-8c01-5232dc99c831: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3c555a25-8b45-41a8-8c01-5232dc99c831', 'volume_size': 1} May 28 21:24:00.346183 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] creating volume 'volume-3c555a25-8b45-41a8-8c01-5232dc99c831' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:00.349022 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:00.424690 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] Created reservations ['db3572b6-d60c-476c-8eff-5ae8f0f330f9', '6faeecd8-f3df-4123-af28-5b7ee7e3de59', '4857a675-7a00-4c70-9ff4-90b57b7a09c5', '999f9835-6a07-4b5c-aece-152018bcbe70'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:24:00.742635 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:00.828620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:00.935850 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:24:00.941104 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:24:00.941946 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:01.089859 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:01.171305 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:24:01.181393 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:01.182031 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:01.188851 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f6cc7722-9865-4aa2-acbd-7766cf06bd71) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3c555a25-8b45-41a8-8c01-5232dc99c831', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:01.202724 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9b56a4dc-29f3-4efd-94f5-d1c79a075b6f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:01.218722 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] Deleted volume successfully. May 28 21:24:01.304490 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Volume volume-3c555a25-8b45-41a8-8c01-5232dc99c831 (3c555a25-8b45-41a8-8c01-5232dc99c831): created successfully May 28 21:24:01.308750 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9b56a4dc-29f3-4efd-94f5-d1c79a075b6f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:01.325825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Flow 'volume_create_manager' (f524795a-ce3f-494b-8b28-f4eb37e80709) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:01.364033 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c88ef855-9382-449d-8809-b7003f92ec1f tempest-VolumesListTestJSON-1352519245 None] Lock "b'cinder-cf706296-6b08-411c-aad7-c87d118555dd-delete_volume'" released by "delete_volume" :: held 4.492s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:24:01.411121 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2cd7d17a-8445-4dba-a7f3-3e67f28e6e70 tempest-VolumeTypesAccessTest-663779615 None] Created volume successfully. May 28 21:24:04.300658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Lock "b'cinder-3c555a25-8b45-41a8-8c01-5232dc99c831-delete_volume'" acquired by "delete_volume" :: waited 0.065s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:24:04.589095 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:04.816312 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:24:04.816986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Volume volume-3c555a25-8b45-41a8-8c01-5232dc99c831 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:24:04.920426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] deleting rbd volume volume-3c555a25-8b45-41a8-8c01-5232dc99c831 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:24:08.193433 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesAccessTest-volume-type-1517756878 is set by the default quota flag: quota_volumes_tempest-VolumeTypesAccessTest-volume-type-1517756878, it is now deprecated. Please use the default quota class for default quota. May 28 21:24:08.195305 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesAccessTest-volume-type-1517756878 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesAccessTest-volume-type-1517756878, it is now deprecated. Please use the default quota class for default quota. May 28 21:24:08.290875 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Created reservations ['c9783aea-e605-45ed-bcc2-0a1e2dc7eaa8', 'd8f6f591-03c9-4051-a19d-d8ab9b5e55b8', '9433efc1-ef53-40f5-a7b5-ac709afb9ce8', 'e3006773-8bed-47c0-8b99-906f79244c91'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:24:08.408022 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:DELETE=1,cinder:SELECT=64,cinder:UPDATE=25,cinder:INSERT=10 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:08.420630 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=52,cinder:UPDATE=38,cinder:DELETE=4,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:08.576835 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:08.649904 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:08.741932 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:24:08.756242 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Deleted volume successfully. May 28 21:24:08.855874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-386056de-6782-4edc-835f-bb580db18d41 tempest-VolumeTypesAccessTest-663779615 None] Lock "b'cinder-3c555a25-8b45-41a8-8c01-5232dc99c831-delete_volume'" released by "delete_volume" :: held 4.555s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:24:22.383561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:22.471095 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Flow 'volume_create_manager' (74a3d2f4-cbf3-41d3-93c1-a0f769b9a89f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:22.514390 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (559e9301-92a6-41d3-aff3-623a245e8cc0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:22.574506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (559e9301-92a6-41d3-aff3-623a245e8cc0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeTypesTest-volume-199851935',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=98d94d82-9602-4efc-97c7-a74fa18057b2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='0fe7e4f1396c4cf7be16906e19cb9a19',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:22Z,use_quota=True,user_id='53840cc1ce544712a9954e3d22d98cfe',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(0b7c759f-fb56-42af-8651-c1d9aab23cd8),volume_type_id=0b7c759f-fb56-42af-8651-c1d9aab23cd8)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:22.584452 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (df98428b-3cef-4996-b955-b39b3cf26893) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:22.596440 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (df98428b-3cef-4996-b955-b39b3cf26893) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:22.610289 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dfbdeaf4-06a7-4c8f-984d-0543ab4db5a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:22.625694 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (dfbdeaf4-06a7-4c8f-984d-0543ab4db5a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '98d94d82-9602-4efc-97c7-a74fa18057b2', 'volume_name': 'volume-98d94d82-9602-4efc-97c7-a74fa18057b2', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:22.642315 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (403b4c4a-9bd0-4a9d-89ee-5131ee66d107) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:22.647575 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (403b4c4a-9bd0-4a9d-89ee-5131ee66d107) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:22.653696 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (95a77046-ad75-4302-bfb6-a30c04d8e3d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:22.655381 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Volume 98d94d82-9602-4efc-97c7-a74fa18057b2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-98d94d82-9602-4efc-97c7-a74fa18057b2', 'volume_size': 1} May 28 21:24:22.656127 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] creating volume 'volume-98d94d82-9602-4efc-97c7-a74fa18057b2' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:22.657207 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:23.894106 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:23.896362 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] extra_specs: {'storage_protocol': 'ceph', 'vendor_name': 'Open Source'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:23.902662 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (95a77046-ad75-4302-bfb6-a30c04d8e3d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-98d94d82-9602-4efc-97c7-a74fa18057b2', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:23.912471 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8c96cc70-f4dd-40a8-9c00-1913987af46d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:24.253677 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Volume volume-98d94d82-9602-4efc-97c7-a74fa18057b2 (98d94d82-9602-4efc-97c7-a74fa18057b2): created successfully May 28 21:24:24.258236 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8c96cc70-f4dd-40a8-9c00-1913987af46d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:24.271128 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Flow 'volume_create_manager' (74a3d2f4-cbf3-41d3-93c1-a0f769b9a89f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:24.398623 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8a168f0a-f246-4eb4-8926-adcebef2acc2 tempest-VolumeTypesTest-2044389601 None] Created volume successfully. May 28 21:24:26.147568 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-303b21fa-c92e-49e7-958d-5e5f099495da tempest-VolumeTypesTest-2044389601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:26.215933 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-303b21fa-c92e-49e7-958d-5e5f099495da tempest-VolumeTypesTest-2044389601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:26.281455 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-303b21fa-c92e-49e7-958d-5e5f099495da tempest-VolumeTypesTest-2044389601 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:24:26.289036 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-303b21fa-c92e-49e7-958d-5e5f099495da tempest-VolumeTypesTest-2044389601 None] Retype volume completed successfully. May 28 21:24:30.546734 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Lock "b'cinder-98d94d82-9602-4efc-97c7-a74fa18057b2-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:24:30.780697 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:31.402996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:24:31.403415 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Volume volume-98d94d82-9602-4efc-97c7-a74fa18057b2 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:24:31.531890 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] deleting rbd volume volume-98d94d82-9602-4efc-97c7-a74fa18057b2 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:24:34.549581 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Deprecated: Default quota for resource: volumes_tempest-VolumeTypesTest-volume-type-1651188183 is set by the default quota flag: quota_volumes_tempest-VolumeTypesTest-volume-type-1651188183, it is now deprecated. Please use the default quota class for default quota. May 28 21:24:34.554629 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Deprecated: Default quota for resource: gigabytes_tempest-VolumeTypesTest-volume-type-1651188183 is set by the default quota flag: quota_gigabytes_tempest-VolumeTypesTest-volume-type-1651188183, it is now deprecated. Please use the default quota class for default quota. May 28 21:24:34.706614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Created reservations ['d9641db1-24b6-4b77-9dcd-2b7c6c19f982', '4434211f-c124-45c4-b5b0-3525aded50f3', 'ee11adb2-e92e-4f6a-aa0c-1db161c7ea36', '8924e910-7aed-40fb-850c-3e8f75514a0e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:24:34.986865 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:35.054761 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:35.150233 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:24:35.161286 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Deleted volume successfully. May 28 21:24:35.272517 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1fafde13-e67d-4cca-a7f2-024c282fef1b tempest-VolumeTypesTest-2044389601 None] Lock "b'cinder-98d94d82-9602-4efc-97c7-a74fa18057b2-delete_volume'" released by "delete_volume" :: held 4.726s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:24:36.055330 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-1828db8b-a902-4ed8-9bf3-be3ec5fd6094 req-9fc2895b-b0c4-41db-9fd6-79274b5e5d0f tempest-AttachVolumeShelveTestJSON-1592071011 None] No connector for attachment a2180126-15a0-4f0a-a05f-9c4706a32383; skipping storage backend terminate_connection call. {{(pid=135921) _connection_terminate /opt/stack/cinder/cinder/volume/manager.py:4914}} May 28 21:24:36.055330 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-1828db8b-a902-4ed8-9bf3-be3ec5fd6094 req-9fc2895b-b0c4-41db-9fd6-79274b5e5d0f tempest-AttachVolumeShelveTestJSON-1592071011 None] Deleting attachment a2180126-15a0-4f0a-a05f-9c4706a32383. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:24:42.529308 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=35,cinder:UPDATE=16,cinder:DELETE=1,cinder:INSERT=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:42.546645 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=22,cinder:SELECT=37,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:24:42.674586 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:42.731103 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Flow 'volume_create_manager' (e3394722-5b26-4525-a07e-8f2aea0e005e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:42.767483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5a3a7c71-b999-4ca2-bfd0-88e378871049) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:42.796186 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5a3a7c71-b999-4ca2-bfd0-88e378871049) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:42Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1258917744',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=46279274-f89b-4df1-a37e-12f9448d2ce6,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='2ce54a0434cb4333a5a7072b01079883',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:42Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:42Z,use_quota=True,user_id='f6ea3fb90c2343ceaaeac6c7efcef26d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:42.803161 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0a7aaf2d-424a-43b6-bcb9-42aa4eddf635) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:42.809714 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0a7aaf2d-424a-43b6-bcb9-42aa4eddf635) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:42.816104 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0c571883-9337-4a1f-9765-96c7ee344a9b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:42.821068 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0c571883-9337-4a1f-9765-96c7ee344a9b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '46279274-f89b-4df1-a37e-12f9448d2ce6', 'volume_name': 'volume-46279274-f89b-4df1-a37e-12f9448d2ce6', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:42.828099 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (45eb4ed1-8817-458f-abfa-026ed03d3e45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:42.832392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (45eb4ed1-8817-458f-abfa-026ed03d3e45) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:42.837736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b2cb9ce5-bd90-4b94-9270-86b5230a1106) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:42.839262 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Volume 46279274-f89b-4df1-a37e-12f9448d2ce6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-46279274-f89b-4df1-a37e-12f9448d2ce6', 'volume_size': 1} May 28 21:24:42.839859 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] creating volume 'volume-46279274-f89b-4df1-a37e-12f9448d2ce6' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:42.840909 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:43.298801 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:43.299395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:43.308017 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b2cb9ce5-bd90-4b94-9270-86b5230a1106) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-46279274-f89b-4df1-a37e-12f9448d2ce6', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:43.312303 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (68e71d7a-2c0e-4cf1-add9-2c9deaf6a993) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:43.461883 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Volume volume-46279274-f89b-4df1-a37e-12f9448d2ce6 (46279274-f89b-4df1-a37e-12f9448d2ce6): created successfully May 28 21:24:43.466815 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (68e71d7a-2c0e-4cf1-add9-2c9deaf6a993) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:43.472565 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Flow 'volume_create_manager' (e3394722-5b26-4525-a07e-8f2aea0e005e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:43.609821 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8d67f02b-4760-40fe-bc64-a504b546644c tempest-VolumesTestJSON-934406337 None] Created volume successfully. May 28 21:24:44.780044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:44.827939 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Flow 'volume_create_manager' (fae3be15-640b-4292-a578-0055b8c4d188) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:44.836415 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (74009579-3ab0-4151-86ca-8b98e0391972) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:44.861506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (74009579-3ab0-4151-86ca-8b98e0391972) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-261606720',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=16d73e07-848d-4a3e-a6ed-ef8d1e25a332,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='52102c097d7b4a81aa28fe0007377926',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:45Z,use_quota=True,user_id='c6be46d2745c45729e2b105052747a02',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:44.868226 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7a1edbd9-d7b1-4970-9765-e758e38d16f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:44.873439 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7a1edbd9-d7b1-4970-9765-e758e38d16f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:44.879083 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bd6bbd29-18cb-40ec-abcf-79c024078184) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:44.883689 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bd6bbd29-18cb-40ec-abcf-79c024078184) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '16d73e07-848d-4a3e-a6ed-ef8d1e25a332', 'volume_name': 'volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:44.888926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d90a181-bc43-41a3-a10d-af36c06322fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:44.893054 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2d90a181-bc43-41a3-a10d-af36c06322fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:44.897920 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68d93721-fb55-4d4d-bef3-806a9d15990b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:44.899214 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Volume 16d73e07-848d-4a3e-a6ed-ef8d1e25a332: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332', 'volume_size': 1} May 28 21:24:44.905723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] creating volume 'volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:44.905723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:45.614575 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:45.614575 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:45.617751 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (68d93721-fb55-4d4d-bef3-806a9d15990b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:45.625884 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (58764928-4cf7-45c6-8d23-1dd05e695365) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:45.741787 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Volume volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332 (16d73e07-848d-4a3e-a6ed-ef8d1e25a332): created successfully May 28 21:24:45.747426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (58764928-4cf7-45c6-8d23-1dd05e695365) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:45.763256 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Flow 'volume_create_manager' (fae3be15-640b-4292-a578-0055b8c4d188) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:45.839382 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-31b0521b-f891-42ae-be6a-7b0bb4b1654f tempest-VolumesListAdminTestJSON-816414189 None] Created volume successfully. May 28 21:24:45.891409 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:45.940463 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Flow 'volume_create_manager' (e3201a7c-cf6b-48b3-b93d-b24ababc96c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:45.964752 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5125b9b1-a4b6-47f6-a0de-e0facffb1baf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:46.013226 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5125b9b1-a4b6-47f6-a0de-e0facffb1baf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-40118408',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='2ce54a0434cb4333a5a7072b01079883',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:46Z,use_quota=True,user_id='f6ea3fb90c2343ceaaeac6c7efcef26d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:46.028660 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1276f3d4-d8e1-48ce-9a0f-78d6200dfde1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:46.029032 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1276f3d4-d8e1-48ce-9a0f-78d6200dfde1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:46.036438 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59616bc6-b47f-4cf8-9b5e-a1c0cc3661dc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:46.045716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59616bc6-b47f-4cf8-9b5e-a1c0cc3661dc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee', 'volume_name': 'volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:46.052402 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1629a737-5973-4a0d-8179-66320c6e60d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:46.059297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1629a737-5973-4a0d-8179-66320c6e60d3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:46.068915 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c1a85144-2b54-422a-84d0-b450e9689253) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:46.070245 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Volume 6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee', 'volume_size': 1} May 28 21:24:46.070773 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] creating volume 'volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:46.071517 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:46.561490 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:46.562008 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:46.565827 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c1a85144-2b54-422a-84d0-b450e9689253) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:46.593807 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (36c9e522-ba2e-4c8b-b6c9-0bdc66f92d1a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:46.727764 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Volume volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee (6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee): created successfully May 28 21:24:46.731527 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (36c9e522-ba2e-4c8b-b6c9-0bdc66f92d1a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:46.742776 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Flow 'volume_create_manager' (e3201a7c-cf6b-48b3-b93d-b24ababc96c5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:46.892212 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dfbd41bf-760d-42f9-9858-210486e4f401 tempest-VolumesTestJSON-934406337 None] Created volume successfully. May 28 21:24:48.047153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:48.114492 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Flow 'volume_create_manager' (e381c2ee-f899-4841-b270-deb925a08b7c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:48.121478 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3f56a580-4070-45e9-947d-d9cad969c83f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:48.194122 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3f56a580-4070-45e9-947d-d9cad969c83f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:47Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-117240535',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='52102c097d7b4a81aa28fe0007377926',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:48Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:48Z,use_quota=True,user_id='c6be46d2745c45729e2b105052747a02',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:48.203078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d66cf5c9-c2f5-4476-a6f7-5e140c4fa459) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:48.209587 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d66cf5c9-c2f5-4476-a6f7-5e140c4fa459) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:48.215421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0ef47a46-563b-4921-92d2-cbee3e865cc1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:48.219734 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0ef47a46-563b-4921-92d2-cbee3e865cc1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f', 'volume_name': 'volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:48.225634 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (33fc9af1-14e5-450b-bfba-b5f8e1e0a181) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:48.231118 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (33fc9af1-14e5-450b-bfba-b5f8e1e0a181) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:48.236728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85e9ba4c-a40f-4a97-aa7e-1d17b24df4cb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:48.238392 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Volume 0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f', 'volume_size': 1} May 28 21:24:48.239767 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] creating volume 'volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:48.240758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:48.842421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:48.843058 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:48.847195 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (85e9ba4c-a40f-4a97-aa7e-1d17b24df4cb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:48.852445 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ecb54991-8d16-4949-9cb1-8a8668c64300) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:48.985670 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Volume volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f (0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f): created successfully May 28 21:24:48.998413 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ecb54991-8d16-4949-9cb1-8a8668c64300) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:49.019555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Flow 'volume_create_manager' (e381c2ee-f899-4841-b270-deb925a08b7c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:49.090394 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8fa6b55c-9cf1-4aa4-a0aa-56bc5acf12dc tempest-VolumesListAdminTestJSON-816414189 None] Created volume successfully. May 28 21:24:49.372516 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:49.419605 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Flow 'volume_create_manager' (be89351d-5702-4883-a8eb-3c4227f8c235) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:49.449474 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b1e4ab2e-99d2-4377-9ef1-996367c585d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:49.490267 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b1e4ab2e-99d2-4377-9ef1-996367c585d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTestJSON-volume-1447808041',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1b3e0b48-0fd1-4613-859e-334ac1076750,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='2ce54a0434cb4333a5a7072b01079883',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:49Z,use_quota=True,user_id='f6ea3fb90c2343ceaaeac6c7efcef26d',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:49.506888 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (41029f94-d627-43d1-a2ba-68f2f79dbeb8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:49.518044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (41029f94-d627-43d1-a2ba-68f2f79dbeb8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:49.533369 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (50455ff0-a073-42cf-b23f-dfc683e04798) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:49.539566 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (50455ff0-a073-42cf-b23f-dfc683e04798) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1b3e0b48-0fd1-4613-859e-334ac1076750', 'volume_name': 'volume-1b3e0b48-0fd1-4613-859e-334ac1076750', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:49.554409 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (acc302ce-f512-4c18-9481-9f3a0c8d4eaf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:49.571364 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (acc302ce-f512-4c18-9481-9f3a0c8d4eaf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:49.586203 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (50f3daac-f75b-41c5-ace2-ec3de43fd9a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:49.586203 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Volume 1b3e0b48-0fd1-4613-859e-334ac1076750: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1b3e0b48-0fd1-4613-859e-334ac1076750', 'volume_size': 1} May 28 21:24:49.589025 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] creating volume 'volume-1b3e0b48-0fd1-4613-859e-334ac1076750' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:49.590014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:50.078811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:50.079476 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:50.084841 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (50f3daac-f75b-41c5-ace2-ec3de43fd9a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1b3e0b48-0fd1-4613-859e-334ac1076750', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:50.091227 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bc51ed9d-f1e8-480c-91b4-02575d28f5de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:50.226837 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Volume volume-1b3e0b48-0fd1-4613-859e-334ac1076750 (1b3e0b48-0fd1-4613-859e-334ac1076750): created successfully May 28 21:24:50.231580 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bc51ed9d-f1e8-480c-91b4-02575d28f5de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:50.245022 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Flow 'volume_create_manager' (be89351d-5702-4883-a8eb-3c4227f8c235) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:50.347392 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-37fb8450-a526-4cbc-83b1-3bab4cf4b658 tempest-VolumesTestJSON-934406337 None] Created volume successfully. May 28 21:24:50.965913 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:50.992581 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Flow 'volume_create_manager' (688b701a-fee5-4c1c-a155-e698ca0ef2e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:51.020518 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e86cd98d-d007-4337-b74f-787ee3f12508) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:51.053097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e86cd98d-d007-4337-b74f-787ee3f12508) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesListAdminTestJSON-Volume-1973258845',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9a42705d-beab-4af5-b3bb-07c791aa52ab,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='52102c097d7b4a81aa28fe0007377926',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:51Z,use_quota=True,user_id='c6be46d2745c45729e2b105052747a02',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:51.073874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0370ae1e-4c32-4a3d-b636-ef2fd11be33b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:51.080021 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0370ae1e-4c32-4a3d-b636-ef2fd11be33b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:51.087004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05369c32-544e-4847-811c-f74f5a1718cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:51.092251 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05369c32-544e-4847-811c-f74f5a1718cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9a42705d-beab-4af5-b3bb-07c791aa52ab', 'volume_name': 'volume-9a42705d-beab-4af5-b3bb-07c791aa52ab', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:51.097473 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f1202aec-60a3-4e02-822b-e4823b3ac64c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:51.105574 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f1202aec-60a3-4e02-822b-e4823b3ac64c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:51.107926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7de205f4-f432-4fff-8f32-a48d10706d24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:51.110233 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Volume 9a42705d-beab-4af5-b3bb-07c791aa52ab: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9a42705d-beab-4af5-b3bb-07c791aa52ab', 'volume_size': 1} May 28 21:24:51.110483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] creating volume 'volume-9a42705d-beab-4af5-b3bb-07c791aa52ab' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:51.111155 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:51.721979 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:51.728494 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:51.728494 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7de205f4-f432-4fff-8f32-a48d10706d24) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9a42705d-beab-4af5-b3bb-07c791aa52ab', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:51.732529 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (612d7b2b-a534-44db-b2b4-e7c791ab2431) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:51.832474 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Volume volume-9a42705d-beab-4af5-b3bb-07c791aa52ab (9a42705d-beab-4af5-b3bb-07c791aa52ab): created successfully May 28 21:24:51.836130 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (612d7b2b-a534-44db-b2b4-e7c791ab2431) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:51.845681 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Flow 'volume_create_manager' (688b701a-fee5-4c1c-a155-e698ca0ef2e2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:51.929799 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-475dcf43-11af-49b7-8785-34c0bab9cf4e tempest-VolumesListAdminTestJSON-816414189 None] Created volume successfully. May 28 21:24:53.716534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] Lock "b'cinder-1b3e0b48-0fd1-4613-859e-334ac1076750-delete_volume'" acquired by "delete_volume" :: waited 0.018s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:24:53.949288 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:54.248277 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:24:54.248712 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] Volume volume-1b3e0b48-0fd1-4613-859e-334ac1076750 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:24:54.309283 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] deleting rbd volume volume-1b3e0b48-0fd1-4613-859e-334ac1076750 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:24:54.856976 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:24:54.918674 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Flow 'volume_create_manager' (abde2a55-22cb-49ea-bcfd-36cc8dcc4c68) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:54.928872 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a98bcbfa-c102-464a-a371-a76749afbda9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:54.975972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a98bcbfa-c102-464a-a371-a76749afbda9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:24:54Z,deleted=False,deleted_at=None,display_description=None,display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=25ea2241-59fa-48c5-be81-bd8c23e43795,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='78b4e47aaec146f8be53d9b3a26b1b6b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:24:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:24:55Z,use_quota=True,user_id='fca31bf977974d349e713cd3b6c09f27',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:54.988571 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (332e47f9-d9f3-4ed1-ad16-480e0cb279ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:54.994281 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (332e47f9-d9f3-4ed1-ad16-480e0cb279ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:55.000708 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7bd11f79-be42-412e-9adf-19ddb67b321f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:55.006404 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7bd11f79-be42-412e-9adf-19ddb67b321f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '25ea2241-59fa-48c5-be81-bd8c23e43795', 'volume_name': 'volume-25ea2241-59fa-48c5-be81-bd8c23e43795', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:55.027685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bfd18d74-3b92-4e42-9f38-2d1e1ac51db9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:55.032851 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bfd18d74-3b92-4e42-9f38-2d1e1ac51db9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:55.039844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a9273fbe-25ae-45a7-84d6-0e0e9fd01c4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:55.042145 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Volume 25ea2241-59fa-48c5-be81-bd8c23e43795: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-25ea2241-59fa-48c5-be81-bd8c23e43795', 'volume_size': 1} May 28 21:24:55.043166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] creating volume 'volume-25ea2241-59fa-48c5-be81-bd8c23e43795' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:24:55.044254 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:55.913223 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:24:55.914777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:24:55.920200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a9273fbe-25ae-45a7-84d6-0e0e9fd01c4a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-25ea2241-59fa-48c5-be81-bd8c23e43795', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:55.926457 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (95b10a34-5653-49f9-95c7-71f5fb79bdbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:24:56.028007 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Volume volume-25ea2241-59fa-48c5-be81-bd8c23e43795 (25ea2241-59fa-48c5-be81-bd8c23e43795): created successfully May 28 21:24:56.034859 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (95b10a34-5653-49f9-95c7-71f5fb79bdbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:24:56.057436 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Flow 'volume_create_manager' (abde2a55-22cb-49ea-bcfd-36cc8dcc4c68) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:24:56.191760 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0f6f28b3-4dd9-44d4-9340-ade6d7350e19 tempest-VolumesListAdminTestJSON-387776628 None] Created volume successfully. May 28 21:24:57.213790 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] Lock "b'cinder-25ea2241-59fa-48c5-be81-bd8c23e43795-delete_volume'" acquired by "delete_volume" :: waited 0.056s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:24:57.406497 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] Created reservations ['ace787ce-f585-41cc-84d6-43c427c1e0b5', '135c139b-786b-436e-a499-b02817278b94', 'bbfb3190-07a6-4ce4-a7e2-50999a22837f', 'ae9509b9-654f-4058-a23c-d98606c16a1b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:24:57.434895 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:57.631961 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:24:57.632399 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] Volume volume-25ea2241-59fa-48c5-be81-bd8c23e43795 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:24:57.739876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] deleting rbd volume volume-25ea2241-59fa-48c5-be81-bd8c23e43795 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:24:57.780045 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] Lock "b'cinder-9a42705d-beab-4af5-b3bb-07c791aa52ab-delete_volume'" acquired by "delete_volume" :: waited 0.025s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:24:57.925084 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:58.088125 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:24:58.088571 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] Volume volume-9a42705d-beab-4af5-b3bb-07c791aa52ab is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:24:58.204096 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] deleting rbd volume volume-9a42705d-beab-4af5-b3bb-07c791aa52ab {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:24:58.212367 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:58.323147 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:58.386067 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:24:58.398151 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] Deleted volume successfully. May 28 21:24:58.514648 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a9049b1e-f55f-4a89-93a7-fb7934ebd272 tempest-VolumesTestJSON-934406337 None] Lock "b'cinder-1b3e0b48-0fd1-4613-859e-334ac1076750-delete_volume'" released by "delete_volume" :: held 4.799s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:24:58.558811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] Lock "b'cinder-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:24:58.734255 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:24:59.026615 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:24:59.027211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] Volume volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:24:59.262582 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] deleting rbd volume volume-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:00.942906 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:25:00.943628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:01.013872 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:01.088161 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:02.436539 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=69,cinder:UPDATE=23,cinder:DELETE=6,cinder:INSERT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:02.620896 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] Created reservations ['1c25eeab-2432-4cf1-b2b6-3d4702c98443', '782aabd3-0344-4fc8-84b0-4c3f195bc4ec', '51dc6304-699c-45f8-a065-e41a84ee06ea', '48b3406e-c3e1-4758-ab10-207a5663eb30'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:02.906900 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:03.041774 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:03.114385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:03.123266 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] Deleted volume successfully. May 28 21:25:03.226681 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-b9c0708c-1e5f-40f5-a22e-65da6b3623b2 tempest-VolumesListAdminTestJSON-387776628 None] Lock "b'cinder-25ea2241-59fa-48c5-be81-bd8c23e43795-delete_volume'" released by "delete_volume" :: held 6.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:03.887087 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=57,cinder:UPDATE=34,cinder:DELETE=3,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:04.011262 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] Created reservations ['2b7557cd-0544-4acf-8fce-5ce26eb48e6f', 'beeb2480-645d-478a-8751-8760687b5456', 'de5c0741-e88a-41b9-b84c-433d6605efd4', '29a878f2-b5da-47f2-8eef-6d37b6223483'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:04.256143 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:04.294522 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:04.360910 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:04.365894 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] Deleted volume successfully. May 28 21:25:04.453234 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6617275b-be2d-48b5-827d-21fdce25796f tempest-VolumesListAdminTestJSON-816414189 None] Lock "b'cinder-9a42705d-beab-4af5-b3bb-07c791aa52ab-delete_volume'" released by "delete_volume" :: held 6.679s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:04.541211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] Created reservations ['c8d587af-577a-4509-9675-e1ca0454a8ce', '12abf46b-3b6b-47de-ba3b-65d59c711aea', '83d7bf42-ebb9-4a1c-858b-192287ef1c19', '4d353421-34e1-45b2-a903-5c35d60b552e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:04.786957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:04.859926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:04.928823 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:04.935414 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] Deleted volume successfully. May 28 21:25:05.014050 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5ebd69a8-dcc8-462b-a7db-643bc0c07993 tempest-VolumesTestJSON-934406337 None] Lock "b'cinder-6d93bb09-25ee-4bb3-9e92-bc6e8f9bd4ee-delete_volume'" released by "delete_volume" :: held 6.455s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:05.448421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] Lock "b'cinder-46279274-f89b-4df1-a37e-12f9448d2ce6-delete_volume'" acquired by "delete_volume" :: waited 0.020s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:25:05.492392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] Lock "b'cinder-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:25:05.554373 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:05.711844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:25:05.713137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] Volume volume-46279274-f89b-4df1-a37e-12f9448d2ce6 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:25:05.809394 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] deleting rbd volume volume-46279274-f89b-4df1-a37e-12f9448d2ce6 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:05.833380 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:06.031893 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:25:06.032490 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] Volume volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:25:06.389598 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] deleting rbd volume volume-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:11.245604 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=57,cinder:INSERT=8,cinder:UPDATE=33,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:11.302404 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] Created reservations ['e18a2daf-677e-4a70-b3ce-1d024fff3589', '355cc417-cbdb-4cd3-8a2c-f8c022cf94df', '1c1aeb7a-4b1b-454d-8663-edfe5b7ac752', '80aaf3ee-fe18-4125-8f7e-7d3edc1e30c2'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:11.380722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] Created reservations ['c13bc65d-1cbf-4c4e-a2ee-fa3ad64a77da', '031c2b1c-3d9c-4491-9ddb-9ab26fd34205', 'd6700d51-350a-4d69-b93a-d51a6558b76e', 'd2be1b97-4fe7-42e9-a985-4abc925c2f58'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:11.547039 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:11.597014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:11.622531 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:11.628512 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] Deleted volume successfully. May 28 21:25:11.718500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:11.768977 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:11.826129 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:11.836050 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] Deleted volume successfully. May 28 21:25:11.841632 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-0d5a6217-48d8-4b49-8fd0-fd099e867523 tempest-VolumesTestJSON-934406337 None] Lock "b'cinder-46279274-f89b-4df1-a37e-12f9448d2ce6-delete_volume'" released by "delete_volume" :: held 6.394s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:11.916381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8d2d79b0-b2ec-4e86-b2a0-7b46137a798d tempest-VolumesListAdminTestJSON-816414189 None] Lock "b'cinder-0dad95f5-4484-4c1a-a4d9-5c9e8a064b6f-delete_volume'" released by "delete_volume" :: held 6.424s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:12.243741 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] Lock "b'cinder-16d73e07-848d-4a3e-a6ed-ef8d1e25a332-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:25:12.393655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:12.506563 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:25:12.509307 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] Volume volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:25:12.610309 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] deleting rbd volume volume-16d73e07-848d-4a3e-a6ed-ef8d1e25a332 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:15.430113 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] Created reservations ['fd1db733-8a80-439c-ac91-43f9494f1d32', '0f9eff07-11d2-4f6e-bc1b-81165cac86df', 'b0fea09b-66c9-416a-a6ff-3c276eb34163', 'b25f406c-03d7-400e-9f63-7d0cd7a79fa1'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:15.719112 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:15.760630 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:15.836014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:15.840829 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] Deleted volume successfully. May 28 21:25:15.928500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-0a0d0592-404b-410c-931f-5bf426fb04a9 tempest-VolumesListAdminTestJSON-816414189 None] Lock "b'cinder-16d73e07-848d-4a3e-a6ed-ef8d1e25a332-delete_volume'" released by "delete_volume" :: held 3.686s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:18.588179 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:25:18.613775 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Flow 'volume_create_manager' (f1059560-0c42-402b-9074-2f23db864396) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:25:18.622678 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e8ab0b2d-d2f6-40fe-8707-3b26d6a040d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:18.682156 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e8ab0b2d-d2f6-40fe-8707-3b26d6a040d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:25:18Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-2080820780',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4ce735b7-0e4f-41cb-917f-48baf2562f1c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='01fa7893b58741ce84a316632efe7882',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:25:18Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:25:18Z,use_quota=True,user_id='ebc8e00965334bd1bb811a0eefd64b84',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:18.699879 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (281d33ee-03c4-4e4e-8746-8d0b226075d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:18.708414 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (281d33ee-03c4-4e4e-8746-8d0b226075d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:18.732616 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3782784-e1ad-490f-a60a-e283990e2dd9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:18.740612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3782784-e1ad-490f-a60a-e283990e2dd9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4ce735b7-0e4f-41cb-917f-48baf2562f1c', 'volume_name': 'volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:18.757158 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cf0d09d8-3e94-4195-8f59-1ca591e30f06) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:18.785693 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cf0d09d8-3e94-4195-8f59-1ca591e30f06) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:18.798996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a622e676-027b-4553-a806-eb4e1f5d2dc3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:18.805069 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Volume 4ce735b7-0e4f-41cb-917f-48baf2562f1c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c', 'volume_size': 1} May 28 21:25:18.807516 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] creating volume 'volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:25:18.811952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:19.429437 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:25:19.430281 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:25:19.436082 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a622e676-027b-4553-a806-eb4e1f5d2dc3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:19.443784 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (77dc0ba6-a177-41dd-b253-f7f581e81110) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:19.522959 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Volume volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c (4ce735b7-0e4f-41cb-917f-48baf2562f1c): created successfully May 28 21:25:19.530770 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (77dc0ba6-a177-41dd-b253-f7f581e81110) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:19.544315 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Flow 'volume_create_manager' (f1059560-0c42-402b-9074-2f23db864396) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:25:19.674264 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-05ba71e1-f5cb-4b5b-a912-ee0ae4999883 tempest-ServersNegativeTestJSON-498936791 None] Created volume successfully. May 28 21:25:22.526770 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:25:22.577924 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Flow 'volume_create_manager' (f628120e-0e44-47f0-a1b4-632da885feb0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:25:22.586573 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c26eb95d-76eb-42a2-a2cb-e1e8e11dd3a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:22.647987 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c26eb95d-76eb-42a2-a2cb-e1e8e11dd3a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:25:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServersNegativeTestJSON-volume-262766537',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cf449889-2f06-41b0-9ec1-bd5ef48126ce,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='01fa7893b58741ce84a316632efe7882',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:25:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:25:22Z,use_quota=True,user_id='ebc8e00965334bd1bb811a0eefd64b84',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:22.667343 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f7990f10-6d87-41c7-a1b1-dac7955ec128) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:22.672171 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f7990f10-6d87-41c7-a1b1-dac7955ec128) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:22.677132 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d51ffa88-2c34-428f-93b0-aa89abebfcab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:22.681792 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d51ffa88-2c34-428f-93b0-aa89abebfcab) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'cf449889-2f06-41b0-9ec1-bd5ef48126ce', 'volume_name': 'volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:22.686875 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (143c6286-5469-4401-b596-40dcf22bdd8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:22.690836 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (143c6286-5469-4401-b596-40dcf22bdd8e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:22.699995 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cc020ddf-da7f-4808-b28c-b6f917aadff9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:22.701327 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Volume cf449889-2f06-41b0-9ec1-bd5ef48126ce: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce', 'volume_size': 1} May 28 21:25:22.701909 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] creating volume 'volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:25:22.702728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:23.174688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:25:23.174688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:25:23.174688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cc020ddf-da7f-4808-b28c-b6f917aadff9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:23.180608 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f7e74b1d-aad9-414a-aff5-4aa5d60be479) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:23.194861 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=49,cinder:INSERT=10,cinder:UPDATE=38,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:23.335265 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Volume volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce (cf449889-2f06-41b0-9ec1-bd5ef48126ce): created successfully May 28 21:25:23.339685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f7e74b1d-aad9-414a-aff5-4aa5d60be479) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:23.346595 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Flow 'volume_create_manager' (f628120e-0e44-47f0-a1b4-632da885feb0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:25:23.454662 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e6123966-16f3-4807-8b26-9ce632450083 tempest-ServersNegativeTestJSON-498936791 None] Created volume successfully. May 28 21:25:24.411323 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:25:24.479218 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Flow 'volume_create_manager' (7847a31f-e06e-471e-b5bb-48e537e44e9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:25:24.490772 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (db915954-09f6-4b79-bca0-7e1b490970a5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:24.515453 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (db915954-09f6-4b79-bca0-7e1b490970a5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:25:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Volume-1297809583',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=3fe26f75-3c61-47e7-b119-e1b004aba4b9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f791936ea1fc49cfb07986f26a47cabe',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:25:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:25:24Z,use_quota=True,user_id='2267c55067d549e79963425bfc21f45c',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:24.529347 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (990749db-34bc-460c-bf2d-01368da59fa4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:24.540758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (990749db-34bc-460c-bf2d-01368da59fa4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:24.545424 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ebc72cc4-d578-4047-8b36-2faa1ff06833) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:24.557285 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ebc72cc4-d578-4047-8b36-2faa1ff06833) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '3fe26f75-3c61-47e7-b119-e1b004aba4b9', 'volume_name': 'volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:24.563075 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a5d8ac55-b36e-49c3-a4c4-fe227a5c4002) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:24.582518 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a5d8ac55-b36e-49c3-a4c4-fe227a5c4002) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:24.616381 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d092edd6-d8b2-4e3e-baea-b04296a71d43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:24.621804 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Volume 3fe26f75-3c61-47e7-b119-e1b004aba4b9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9', 'volume_size': 1} May 28 21:25:24.629711 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] creating volume 'volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:25:24.630759 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:25.103964 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:25:25.103964 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:25:25.106625 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d092edd6-d8b2-4e3e-baea-b04296a71d43) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:25.114009 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d9aeedae-5adf-449f-8d8b-fba186a522db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:25:25.208480 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Volume volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9 (3fe26f75-3c61-47e7-b119-e1b004aba4b9): created successfully May 28 21:25:25.214423 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d9aeedae-5adf-449f-8d8b-fba186a522db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:25:25.220410 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Flow 'volume_create_manager' (7847a31f-e06e-471e-b5bb-48e537e44e9f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:25:25.340237 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-284957d7-42a5-4643-a0e0-3423dd291baa tempest-VolumesBackupsV39Test-1074139284 None] Created volume successfully. May 28 21:25:27.081933 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:25:26Z,data_timestamp=2022-05-28T21:25:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-1353229858',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=a5804b82-1246-4fb7-89ec-ca69ce549f7c,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='f791936ea1fc49cfb07986f26a47cabe',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='2267c55067d549e79963425bfc21f45c',volume_id=3fe26f75-3c61-47e7-b119-e1b004aba4b9) May 28 21:25:27.094055 ncdv-master-1 cinder-volume[135921]: INFO cinder.rpc [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Automatically selected cinder-backup objects version 1.39 as minimum service version. May 28 21:25:27.109253 ncdv-master-1 cinder-volume[135921]: INFO cinder.rpc [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Automatically selected cinder-backup RPC version 2.3 as minimum service version. May 28 21:25:27.110074 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] continue_backup in rpcapi backup_id a5804b82-1246-4fb7-89ec-ca69ce549f7c {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:25:34.873927 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:25:36.593791 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.720s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:25:36.594781 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '3fe26f75-3c61-47e7-b119-e1b004aba4b9', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:25:36.612698 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Initialize volume connection completed successfully. May 28 21:25:42.338851 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Terminate volume connection completed successfully. May 28 21:25:42.396445 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ff84e19c-a885-4d90-97f5-0c9c7ea0513c tempest-VolumesBackupsV39Test-1074139284 None] Remove volume export completed successfully. May 28 21:25:45.304194 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=41,cinder:UPDATE=19,cinder:INSERT=2,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:50.542679 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] Lock "b'cinder-bd319c3c-84f3-4f60-b04f-30c14dbe419f-delete_volume'" acquired by "delete_volume" :: waited 0.022s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:25:50.641642 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] Lock "b'cinder-3fe26f75-3c61-47e7-b119-e1b004aba4b9-delete_volume'" acquired by "delete_volume" :: waited 0.027s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:25:50.763194 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:51.082379 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:25:51.082820 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:25:51.174396 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] deleting rbd volume volume-bd319c3c-84f3-4f60-b04f-30c14dbe419f {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:51.209110 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:54.142429 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] Volume volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:25:54.614821 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] deleting rbd volume volume-3fe26f75-3c61-47e7-b119-e1b004aba4b9 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:54.894117 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] Created reservations ['1af68f01-2764-44da-a0f8-430cb6b767a6', 'f9ddc7d9-03d2-49dd-ae58-90ab6dbe3f90', 'ad220f2a-1a70-4166-8160-0c28f24e6df1', 'd4055e13-de12-44f4-ab18-a02d930dfb71'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:55.323781 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:55.365402 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:55.395250 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:55.400875 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] Deleted volume successfully. May 28 21:25:55.501234 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-c2586aec-adcd-4eca-ad8f-8bab542b8257 tempest-AttachVolumeShelveTestJSON-1592071011 None] Lock "b'cinder-bd319c3c-84f3-4f60-b04f-30c14dbe419f-delete_volume'" released by "delete_volume" :: held 4.958s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:56.365663 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] Lock "b'cinder-aca16bed-3410-4df8-b98d-9a6854483c48-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:25:56.378128 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=31,cinder:UPDATE=10,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:25:56.516856 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:56.636200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:25:56.636865 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] Volume volume-aca16bed-3410-4df8-b98d-9a6854483c48 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:25:56.843367 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] deleting rbd volume volume-aca16bed-3410-4df8-b98d-9a6854483c48 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:25:58.078340 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] Created reservations ['2ff57b31-c09a-4cea-9e47-a8677b2550dd', 'c6d5fff6-721f-48fb-bb55-83700976e8e1', 'ce3e4690-2d6a-4098-93bd-f60463a5d104', '1c12679d-46d1-48af-a84f-f5977d136af2'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:58.385784 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:58.447884 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:25:58.561109 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:25:58.573035 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] Deleted volume successfully. May 28 21:25:58.693421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a36b37c9-3a20-4af9-a98e-03fc8e5f2477 tempest-VolumesBackupsV39Test-1074139284 None] Lock "b'cinder-3fe26f75-3c61-47e7-b119-e1b004aba4b9-delete_volume'" released by "delete_volume" :: held 8.051s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:25:59.911110 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] Created reservations ['7fd360e6-08b5-4ec8-841f-20d3baa75e0f', 'b5d1d66f-26d7-4197-bfb3-4068ec31b66a', '1b1d0a79-2606-4f0b-b1c8-3c47c52ca65d', '5f31ed92-0560-403d-a566-f2773728f380'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:25:59.928148 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=54,cinder:UPDATE=33,cinder:INSERT=12,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:00.191526 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:00.227768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:00.276271 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:00.280463 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] Deleted volume successfully. May 28 21:26:00.413680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-09d8ca76-bdac-4775-a5cc-87b4677c8501 tempest-AttachVolumeShelveTestJSON-1592071011 None] Lock "b'cinder-aca16bed-3410-4df8-b98d-9a6854483c48-delete_volume'" released by "delete_volume" :: held 4.047s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:26:00.944351 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:26:00.945698 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:01.001536 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:01.049707 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:02.901513 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:02.938528 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (4b160b5c-72e5-4f35-bab9-79f1878ab663) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:02.953643 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0849c5e5-8490-472b-a883-f355e1a0895f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:02.971280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0849c5e5-8490-472b-a883-f355e1a0895f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1157561347',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f21114e7-185b-4f50-9177-b5bdc86ab943,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='13837973df4240769de72a63520a0cbd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:03Z,use_quota=True,user_id='98f2a78a037c417ea3cff70dab6b0e99',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:02.978260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (96cdc131-4f48-473e-b1aa-b90d3ed97cbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:02.983448 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (96cdc131-4f48-473e-b1aa-b90d3ed97cbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:02.990211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bdc1b102-b37c-4f9d-9170-a6e48271df15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:03.129736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bdc1b102-b37c-4f9d-9170-a6e48271df15) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'f21114e7-185b-4f50-9177-b5bdc86ab943', 'volume_name': 'volume-f21114e7-185b-4f50-9177-b5bdc86ab943', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:03.135534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (789cc881-a8d7-45cb-b006-02fd70fc33be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:03.140676 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (789cc881-a8d7-45cb-b006-02fd70fc33be) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:03.145931 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (06479bc4-301c-43a9-852a-ca698b6ddacf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:03.147229 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Volume f21114e7-185b-4f50-9177-b5bdc86ab943: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-f21114e7-185b-4f50-9177-b5bdc86ab943', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:26:03.148557 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Cloning f21114e7-185b-4f50-9177-b5bdc86ab943 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:26:03.159141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:26:03.245942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 26, 3, 164823)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:26:03.245942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:26:03.249606 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:26:03.255341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.096s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:26:03.256706 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:26:03.264421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 26, 3, 260531)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:26:03.265547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:26:03.266213 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:26:03.285047 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:03.430379 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] creating snapshot='volume-f21114e7-185b-4f50-9177-b5bdc86ab943.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:26:05.518198 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-f21114e7-185b-4f50-9177-b5bdc86ab943.clone_snap' to 'volume-f21114e7-185b-4f50-9177-b5bdc86ab943' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:26:06.554957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:26:06.557359 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:06.558027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:06.845723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:26:06.848361 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Marking volume f21114e7-185b-4f50-9177-b5bdc86ab943 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:26:06.959982 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to f21114e7-185b-4f50-9177-b5bdc86ab943. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:26:06.959982 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Creating volume glance metadata for volume f21114e7-185b-4f50-9177-b5bdc86ab943 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:26:07.091169 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (06479bc4-301c-43a9-852a-ca698b6ddacf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f21114e7-185b-4f50-9177-b5bdc86ab943', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:07.101734 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ecf36ad-39cd-4310-84a5-3cdc4f1b7975) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:07.190482 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Volume volume-f21114e7-185b-4f50-9177-b5bdc86ab943 (f21114e7-185b-4f50-9177-b5bdc86ab943): created successfully May 28 21:26:07.195619 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ecf36ad-39cd-4310-84a5-3cdc4f1b7975) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:07.201586 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (4b160b5c-72e5-4f35-bab9-79f1878ab663) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:07.259146 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-4d33ee95-2e38-4d60-9252-b69cbcf7a3c1 tempest-VolumesCloneTest-1257877467 None] Created volume successfully. May 28 21:26:08.815094 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:08.863644 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (d9ab7d74-d619-49f2-afbb-65d25b9652fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:08.863644 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ef209fef-6715-42b1-9941-e2e13b59d2aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:08.873428 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ef209fef-6715-42b1-9941-e2e13b59d2aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-340880902',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=7db03212-cac8-409d-bc55-2a7c2102d4e2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='13837973df4240769de72a63520a0cbd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=f21114e7-185b-4f50-9177-b5bdc86ab943,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:09Z,use_quota=True,user_id='98f2a78a037c417ea3cff70dab6b0e99',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:08.879852 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b16ef9e-7024-4566-bce5-a267abf7efb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:08.884442 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0b16ef9e-7024-4566-bce5-a267abf7efb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:08.890144 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff606b35-af33-4c74-bd39-31ce13f118c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:08.914758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ff606b35-af33-4c74-bd39-31ce13f118c3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '7db03212-cac8-409d-bc55-2a7c2102d4e2', 'volume_name': 'volume-7db03212-cac8-409d-bc55-2a7c2102d4e2', 'volume_size': 1, 'source_volid': 'f21114e7-185b-4f50-9177-b5bdc86ab943', 'source_volstatus': 'available'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:08.921019 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b24a2fb7-5514-4e35-a97e-e31f6a92bfd5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:08.926533 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b24a2fb7-5514-4e35-a97e-e31f6a92bfd5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:08.931934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (842491ce-d0ec-47b0-b2ca-bf21022f68b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:08.933069 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Volume 7db03212-cac8-409d-bc55-2a7c2102d4e2: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-7db03212-cac8-409d-bc55-2a7c2102d4e2', 'volume_size': 1, 'source_volid': 'f21114e7-185b-4f50-9177-b5bdc86ab943', 'source_volstatus': 'available'} May 28 21:26:08.948656 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:09.092774 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] creating snapshot='volume-7db03212-cac8-409d-bc55-2a7c2102d4e2.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:26:10.122991 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] cloning 'volume-f21114e7-185b-4f50-9177-b5bdc86ab943@volume-7db03212-cac8-409d-bc55-2a7c2102d4e2.clone_snap' to 'volume-7db03212-cac8-409d-bc55-2a7c2102d4e2' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:26:10.361011 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-243e3597-0545-4f85-baa7-5149aca4c8ad tempest-GroupSnapshotsTest-1289745910 None] Group tempest-GroupSnapshotsTest-Group-1444235751: creating May 28 21:26:10.446333 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-243e3597-0545-4f85-baa7-5149aca4c8ad tempest-GroupSnapshotsTest-1289745910 None] Group tempest-GroupSnapshotsTest-Group-1444235751: created successfully May 28 21:26:10.455803 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-243e3597-0545-4f85-baa7-5149aca4c8ad tempest-GroupSnapshotsTest-1289745910 None] Create group completed successfully. May 28 21:26:11.106489 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:26:11.108029 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:11.108742 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:11.486895 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:26:11.487538 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Marking volume 7db03212-cac8-409d-bc55-2a7c2102d4e2 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:26:11.584702 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Copying metadata from source volume f21114e7-185b-4f50-9177-b5bdc86ab943 to 7db03212-cac8-409d-bc55-2a7c2102d4e2. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:438}} May 28 21:26:11.685671 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (842491ce-d0ec-47b0-b2ca-bf21022f68b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-7db03212-cac8-409d-bc55-2a7c2102d4e2', 'volume_size': 1, 'source_volid': 'f21114e7-185b-4f50-9177-b5bdc86ab943', 'source_volstatus': 'available'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:11.690564 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1bf3d264-27cd-4724-93aa-0694717b465f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:11.773721 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Volume volume-7db03212-cac8-409d-bc55-2a7c2102d4e2 (7db03212-cac8-409d-bc55-2a7c2102d4e2): created successfully May 28 21:26:11.777299 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (1bf3d264-27cd-4724-93aa-0694717b465f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:11.790653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (d9ab7d74-d619-49f2-afbb-65d25b9652fe) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:11.886621 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-22abadb6-7111-49e1-9ec2-1ce1aee2e556 tempest-VolumesCloneTest-1257877467 None] Created volume successfully. May 28 21:26:12.548451 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:12.573986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (a00a96a7-c2d7-4b5c-9993-e258e9840ac9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:12.576577 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ee4f45ac-64ce-4f75-ae26-315c2549d94d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:12.598207 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ee4f45ac-64ce-4f75-ae26-315c2549d94d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1121249889',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=08b1e13c-c379-4b4b-9410-2fffdec6162e,host='ncdv-master-1@ceph#ceph',id=a45da818-2e9b-4b76-8f17-80e5972d4c77,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='aaed16ff29234c81b161434120a4bf12',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:12Z,use_quota=True,user_id='41c3af0358084a9da8f56f0b37ed7834',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(a0afd0d8-2992-4f91-bbba-7f37595bedfa),volume_type_id=a0afd0d8-2992-4f91-bbba-7f37595bedfa)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:12.605202 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c7931740-58c3-4558-92dc-8033bdd6a021) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:12.612887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c7931740-58c3-4558-92dc-8033bdd6a021) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:12.618398 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (338b8043-e8f7-48bd-bdd5-74b579ed3451) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:12.622660 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (338b8043-e8f7-48bd-bdd5-74b579ed3451) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a45da818-2e9b-4b76-8f17-80e5972d4c77', 'volume_name': 'volume-a45da818-2e9b-4b76-8f17-80e5972d4c77', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:12.628709 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0e3f006e-53e8-4f96-9625-469269537125) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:12.633219 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0e3f006e-53e8-4f96-9625-469269537125) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:12.638941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a8c13b4c-348c-460e-8935-2c665d945f8d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:12.640270 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Volume a45da818-2e9b-4b76-8f17-80e5972d4c77: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a45da818-2e9b-4b76-8f17-80e5972d4c77', 'volume_size': 1} May 28 21:26:12.640671 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] creating volume 'volume-a45da818-2e9b-4b76-8f17-80e5972d4c77' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:26:12.641456 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:12.988212 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:12.988581 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:12.994510 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a8c13b4c-348c-460e-8935-2c665d945f8d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a45da818-2e9b-4b76-8f17-80e5972d4c77', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:13.001799 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3016e601-89c5-474b-bca9-8d3eca42a42d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:13.104802 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Volume volume-a45da818-2e9b-4b76-8f17-80e5972d4c77 (a45da818-2e9b-4b76-8f17-80e5972d4c77): created successfully May 28 21:26:13.109430 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3016e601-89c5-474b-bca9-8d3eca42a42d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:13.116166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (a00a96a7-c2d7-4b5c-9993-e258e9840ac9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:13.258577 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9b32edbc-e1fc-441c-855f-180d9d2a103b tempest-GroupSnapshotsTest-1289745910 None] Created volume successfully. May 28 21:26:13.837643 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:13.851993 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (412755af-37c3-44e6-be43-e99aaa466fb0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:13.860054 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (585efb78-69b9-4e77-a42c-11aa6cfb6c47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:13.874102 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (585efb78-69b9-4e77-a42c-11aa6cfb6c47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1085998280',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=53aaca19-e01e-4f3a-919a-5cb114426a96,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='13837973df4240769de72a63520a0cbd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:14Z,use_quota=True,user_id='98f2a78a037c417ea3cff70dab6b0e99',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:13.879870 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f8ae55a7-2eed-4871-864f-c8f0edb6d203) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:13.884598 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f8ae55a7-2eed-4871-864f-c8f0edb6d203) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:13.889628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c48d84b2-6154-4df0-84fb-b544ab7087ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:13.893699 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c48d84b2-6154-4df0-84fb-b544ab7087ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '53aaca19-e01e-4f3a-919a-5cb114426a96', 'volume_name': 'volume-53aaca19-e01e-4f3a-919a-5cb114426a96', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:13.899209 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0a5cb555-2a95-4797-8835-f6551ae8a071) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:13.903113 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0a5cb555-2a95-4797-8835-f6551ae8a071) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:13.907871 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (df48417c-7d44-4962-97dd-17fc4ee1322c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:13.909018 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Volume 53aaca19-e01e-4f3a-919a-5cb114426a96: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-53aaca19-e01e-4f3a-919a-5cb114426a96', 'volume_size': 1} May 28 21:26:13.910591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] creating volume 'volume-53aaca19-e01e-4f3a-919a-5cb114426a96' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:26:13.910591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:14.051003 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f22e1ce7-1c2d-4f78-b669-0078994b6162 tempest-GroupSnapshotsTest-1289745910 None] GroupSnapshot abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f: creating. May 28 21:26:14.094459 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f22e1ce7-1c2d-4f78-b669-0078994b6162 tempest-GroupSnapshotsTest-1289745910 None] Group snapshot abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f: creating. {{(pid=135921) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4082}} May 28 21:26:14.112145 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f22e1ce7-1c2d-4f78-b669-0078994b6162 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:14.412300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:14.412957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:14.417688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (df48417c-7d44-4962-97dd-17fc4ee1322c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-53aaca19-e01e-4f3a-919a-5cb114426a96', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:14.421545 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4b93caa7-e5d1-4b3f-836e-224e3cefd0af) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:14.527400 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Volume volume-53aaca19-e01e-4f3a-919a-5cb114426a96 (53aaca19-e01e-4f3a-919a-5cb114426a96): created successfully May 28 21:26:14.531231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4b93caa7-e5d1-4b3f-836e-224e3cefd0af) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:14.537080 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (412755af-37c3-44e6-be43-e99aaa466fb0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:14.653416 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5d12fb1e-f463-40ef-875d-b6034b6842d2 tempest-VolumesCloneTest-1257877467 None] Created volume successfully. May 28 21:26:16.129783 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f22e1ce7-1c2d-4f78-b669-0078994b6162 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f: created successfully May 28 21:26:16.618084 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:16.656061 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (4183da3f-30f9-437e-b395-d7b32107eb44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:16.664740 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (23eef572-fa13-42b6-abfb-e40175f4e928) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:16.684195 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (23eef572-fa13-42b6-abfb-e40175f4e928) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1714492347',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=62467732-ebb2-441e-99ea-f25cac47ec0e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='13837973df4240769de72a63520a0cbd',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:16Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=53aaca19-e01e-4f3a-919a-5cb114426a96,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:16Z,use_quota=True,user_id='98f2a78a037c417ea3cff70dab6b0e99',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:16.689811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e326c00f-adaa-4e03-9438-09aa4ef6cb3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:16.694199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e326c00f-adaa-4e03-9438-09aa4ef6cb3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:16.699604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ecc84187-9470-4f68-9877-53e602465856) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:16.716004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ecc84187-9470-4f68-9877-53e602465856) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '62467732-ebb2-441e-99ea-f25cac47ec0e', 'volume_name': 'volume-62467732-ebb2-441e-99ea-f25cac47ec0e', 'volume_size': 2, 'source_volid': '53aaca19-e01e-4f3a-919a-5cb114426a96', 'source_volstatus': 'available'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:16.720986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c24d1213-6243-4715-a453-8e457936ba39) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:16.725362 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c24d1213-6243-4715-a453-8e457936ba39) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:16.731666 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (92ad829e-f0c0-4993-af4a-10ef3871dff9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:16.732821 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Volume 62467732-ebb2-441e-99ea-f25cac47ec0e: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-62467732-ebb2-441e-99ea-f25cac47ec0e', 'volume_size': 2, 'source_volid': '53aaca19-e01e-4f3a-919a-5cb114426a96', 'source_volstatus': 'available'} May 28 21:26:16.744053 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=58,cinder:UPDATE=27,cinder:DELETE=3,cinder:INSERT=12 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:16.746451 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:16.852596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] creating snapshot='volume-62467732-ebb2-441e-99ea-f25cac47ec0e.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:26:18.041940 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] cloning 'volume-53aaca19-e01e-4f3a-919a-5cb114426a96@volume-62467732-ebb2-441e-99ea-f25cac47ec0e.clone_snap' to 'volume-62467732-ebb2-441e-99ea-f25cac47ec0e' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:26:18.119148 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] cloning volumes/volume-a45da818-2e9b-4b76-8f17-80e5972d4c77@snapshot-3c8dd4ef-2f08-48d3-a255-62c3bf96596a to volume-fcecad37-329a-4acd-a2d8-3680a59d8c17 {{(pid=135921) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1098}} May 28 21:26:18.120282 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:18.180244 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:19.329354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Volume volume-53aaca19-e01e-4f3a-919a-5cb114426a96 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:26:19.330563 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:19.330942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:19.768095 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] resize volume 'volume-62467732-ebb2-441e-99ea-f25cac47ec0e' from 1 to 2 {{(pid=135921) _extend_if_required /opt/stack/cinder/cinder/volume/drivers/rbd.py:749}} May 28 21:26:19.768757 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:19.894921 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:19.895297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:19.900321 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:19.942691 ncdv-master-1 cinder-volume[135921]: WARNING cinder.volume.drivers.rbd [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues: NotImplementedError May 28 21:26:20.428229 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e7d67c5d-e4ce-439b-837f-a0573e1a84e2 tempest-GroupSnapshotsTest-1289745910 None] Create group from source-snapshot-abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f completed successfully. May 28 21:26:20.752201 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:26:20.756164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (92ad829e-f0c0-4993-af4a-10ef3871dff9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-62467732-ebb2-441e-99ea-f25cac47ec0e', 'volume_size': 2, 'source_volid': '53aaca19-e01e-4f3a-919a-5cb114426a96', 'source_volstatus': 'available'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:20.763822 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (389ff295-e12f-474c-8735-275faa3644ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:20.874128 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Volume volume-62467732-ebb2-441e-99ea-f25cac47ec0e (62467732-ebb2-441e-99ea-f25cac47ec0e): created successfully May 28 21:26:20.893501 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (389ff295-e12f-474c-8735-275faa3644ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:20.904716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Flow 'volume_create_manager' (4183da3f-30f9-437e-b395-d7b32107eb44) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:21.018806 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f3ea3d7e-b697-487f-ad5d-edd51a515b41 tempest-VolumesCloneTest-1257877467 None] Created volume successfully. May 28 21:26:21.219147 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=77,cinder:UPDATE=8,cinder:INSERT=12,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:21.246688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:21.389526 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:26:21.486932 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] deleting rbd volume volume-fcecad37-329a-4acd-a2d8-3680a59d8c17 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:26:21.939886 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-62467732-ebb2-441e-99ea-f25cac47ec0e-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:26:22.073032 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:22.253108 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:26:22.477198 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] deleting rbd volume volume-62467732-ebb2-441e-99ea-f25cac47ec0e {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:26:26.371594 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['d70bf8f0-16a8-4ee8-92b9-7b983b85b963'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:26.406063 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1583347041 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1583347041, it is now deprecated. Please use the default quota class for default quota. May 28 21:26:26.406944 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1583347041 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1583347041, it is now deprecated. Please use the default quota class for default quota. May 28 21:26:26.487877 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['79fc0bc3-897c-45fd-8684-c904d08a9193', '18182302-cebe-434c-943a-7dc5049708e3', 'f1e893e5-86a4-4965-978b-b40c0522d1b4', 'fc6f83d4-23d7-44ee-be3c-68d70eebb865'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:26.928013 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:26:26.973153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] Volume volume-53aaca19-e01e-4f3a-919a-5cb114426a96 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:26:26.973655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] deleting parent snapshot volume-62467732-ebb2-441e-99ea-f25cac47ec0e.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:26:29.447260 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-df68a92c-c2ea-4677-a6f9-a6c34dfc3a22 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f: deleting May 28 21:26:29.512280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-df68a92c-c2ea-4677-a6f9-a6c34dfc3a22 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f: deleting {{(pid=135921) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4257}} May 28 21:26:29.546347 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:29.597034 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:29.637442 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:29.655011 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-df68a92c-c2ea-4677-a6f9-a6c34dfc3a22 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:29.801887 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dd8c8121-a75d-4515-9892-8c064497a70d tempest-GroupSnapshotsTest-1289745910 None] Delete group completed successfully. May 28 21:26:29.914677 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] Created reservations ['77819b29-438c-45eb-bcc1-92cc1bab16d9', '4b63a1f5-408c-4a30-a16d-bf87272529fe', 'd04e3439-c66a-485f-ab53-9cc4bd77c12d', '4397e9be-9306-4f09-a514-be29c9859ecc'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:30.146233 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:30.180052 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:30.233249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:30.237902 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] Deleted volume successfully. May 28 21:26:30.363007 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-cfa9cde3-61e0-4bde-b054-cf059b8960d0 tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-62467732-ebb2-441e-99ea-f25cac47ec0e-delete_volume'" released by "delete_volume" :: held 8.423s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:26:30.855435 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-53aaca19-e01e-4f3a-919a-5cb114426a96-delete_volume'" acquired by "delete_volume" :: waited 0.055s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:26:31.074674 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:31.253957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:26:31.254656 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] Volume volume-53aaca19-e01e-4f3a-919a-5cb114426a96 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:26:31.338972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] deleting rbd volume volume-53aaca19-e01e-4f3a-919a-5cb114426a96 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:26:32.904941 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-df68a92c-c2ea-4677-a6f9-a6c34dfc3a22 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1583347041 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1583347041, it is now deprecated. Please use the default quota class for default quota. May 28 21:26:33.099234 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-df68a92c-c2ea-4677-a6f9-a6c34dfc3a22 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['00e3809a-75be-45aa-88c5-1f61bd3b390d', '34606fa6-f84e-4b64-9e32-3de882407d96', 'ca35d5ab-3a54-44ae-a714-6e42b29772a3', 'b20f77eb-57c9-43c0-866a-21d2ef876456'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:33.449873 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-df68a92c-c2ea-4677-a6f9-a6c34dfc3a22 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot abd6370b-0d1e-49d1-ac7a-4d4dd3b1f80f: deleted successfully May 28 21:26:34.719688 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=64,cinder:UPDATE=28,cinder:INSERT=7,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:34.762084 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] Created reservations ['9acc1658-25f1-459a-ba3e-762e1fb63ea9', 'bf39acec-e17b-4672-9152-b9813f8e1a56', 'e6c5a04b-f231-4b57-b3a6-9059accef5f4', 'c4dcecd1-b49b-4506-8a4b-512ebe7359f6'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:34.777753 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=57,cinder:UPDATE=33,cinder:INSERT=10 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:34.992149 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:35.258015 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:26:35.258445 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-a45da818-2e9b-4b76-8f17-80e5972d4c77 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:26:35.396857 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] deleting rbd volume volume-a45da818-2e9b-4b76-8f17-80e5972d4c77 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:26:35.399383 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:35.481651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:35.689128 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:35.713817 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] Deleted volume successfully. May 28 21:26:35.732099 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-7db03212-cac8-409d-bc55-2a7c2102d4e2-delete_volume'" acquired by "delete_volume" :: waited 0.035s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:26:35.862485 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e5f7e88a-4f7a-445b-989f-bd6bb81c6b7e tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-53aaca19-e01e-4f3a-919a-5cb114426a96-delete_volume'" released by "delete_volume" :: held 5.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:26:36.104532 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:36.355280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:26:36.918132 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] deleting rbd volume volume-7db03212-cac8-409d-bc55-2a7c2102d4e2 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:26:41.864536 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['a6d94950-8151-4297-9611-20a6b361a627'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:41.939606 ncdv-master-1 cinder-volume[135921]: WARNING cinder.db.sqlalchemy.api [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] Reservation would make usage less than 0 for the following resources, so on commit they will be limited to prevent going below 0: ['gigabytes', 'gigabytes_tempest-GroupSnapshotsTest-volume-type-1583347041'] May 28 21:26:42.051471 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['7104a000-521f-413a-ae97-83d14902945a', '73b82f62-c8d9-47f6-9a36-ea231b9acc4c', 'f597136c-fbb2-4a92-bba9-c07c73854a3e', 'cf6545c1-e122-438b-b495-50091f95a2c1'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:42.197686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:26:42.396300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] deleting parent snapshot volume-7db03212-cac8-409d-bc55-2a7c2102d4e2.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:26:45.192439 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] Created reservations ['192525a3-3fb3-43ce-8b03-79a23215ab34', '82011941-ff66-4827-92b0-4c713e166f36', '343236ac-5b25-4a21-8202-13a232b082ef', '0f579497-f373-43d7-9488-5ff6eeb5f148'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:45.229565 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:45.299927 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:45.337521 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:45.357276 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c97e1b94-7964-406d-a789-1d7cd3054df2 tempest-GroupSnapshotsTest-1289745910 None] Delete group completed successfully. May 28 21:26:45.492593 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:45.617622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:45.678923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:45.687886 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] Deleted volume successfully. May 28 21:26:45.794056 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f5c8d933-6ff9-4606-88f2-729883570148 tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-7db03212-cac8-409d-bc55-2a7c2102d4e2-delete_volume'" released by "delete_volume" :: held 10.062s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:26:46.750221 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-f21114e7-185b-4f50-9177-b5bdc86ab943-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:26:46.898068 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:47.063591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:26:47.161972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] deleting rbd volume volume-f21114e7-185b-4f50-9177-b5bdc86ab943 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:26:47.188979 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-48fcb864-e573-4bfe-a296-d91b340ff141 tempest-GroupSnapshotsTest-1289745910 None] Group tempest-GroupSnapshotsTest-Group-1573420632: creating May 28 21:26:47.286250 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-48fcb864-e573-4bfe-a296-d91b340ff141 tempest-GroupSnapshotsTest-1289745910 None] Group tempest-GroupSnapshotsTest-Group-1573420632: created successfully May 28 21:26:47.297176 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-48fcb864-e573-4bfe-a296-d91b340ff141 tempest-GroupSnapshotsTest-1289745910 None] Create group completed successfully. May 28 21:26:49.293511 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:49.316614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (882de97e-d198-493a-905c-cab833f374b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:49.323423 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3fe3a213-c3be-48bd-a076-436b1ed87200) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:49.339997 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3fe3a213-c3be-48bd-a076-436b1ed87200) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:48Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-1196162763',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=ecc91c7d-b5d2-4c67-b67f-7ab4a1c810c6,host='ncdv-master-1@ceph#ceph',id=8c6b36aa-4f0f-44ae-839a-ca262e615ee0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='aaed16ff29234c81b161434120a4bf12',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:49Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:49Z,use_quota=True,user_id='41c3af0358084a9da8f56f0b37ed7834',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(7496bac0-cc47-4cbf-96fb-5e89149e8a02),volume_type_id=7496bac0-cc47-4cbf-96fb-5e89149e8a02)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:49.345428 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (357e58c2-3323-4fc9-aad0-fcc9b03e8bca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:49.352593 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (357e58c2-3323-4fc9-aad0-fcc9b03e8bca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:49.357882 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aec38d80-9cd6-4295-ba6b-f2fe8efc3a63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:49.362017 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (aec38d80-9cd6-4295-ba6b-f2fe8efc3a63) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8c6b36aa-4f0f-44ae-839a-ca262e615ee0', 'volume_name': 'volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:49.367576 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6be1d4f-8a99-46fe-853a-33e4caf58ed5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:49.371545 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a6be1d4f-8a99-46fe-853a-33e4caf58ed5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:49.376216 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2ffcd379-0f6e-47bc-8073-71595adf5880) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:49.378261 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Volume 8c6b36aa-4f0f-44ae-839a-ca262e615ee0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0', 'volume_size': 1} May 28 21:26:49.378553 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] creating volume 'volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:26:49.379247 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:50.231690 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:50.232355 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:50.236216 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2ffcd379-0f6e-47bc-8073-71595adf5880) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:50.241509 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b64e92e5-0f36-4cbf-88e0-47309a96595c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:50.331345 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0 (8c6b36aa-4f0f-44ae-839a-ca262e615ee0): created successfully May 28 21:26:50.335173 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b64e92e5-0f36-4cbf-88e0-47309a96595c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:50.340658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (882de97e-d198-493a-905c-cab833f374b6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:50.361089 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:26:50.522461 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] deleting parent snapshot volume-f21114e7-185b-4f50-9177-b5bdc86ab943.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:26:53.151801 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1c80a1c8-4a0c-4105-9e6d-3700d945c867 tempest-GroupSnapshotsTest-1289745910 None] Created volume successfully. May 28 21:26:53.173447 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=30,cinder:SELECT=62,cinder:DELETE=3,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:53.231733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:26:53.251429 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (e1c6f995-5114-4a3b-9291-a7fef67b6914) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:53.258206 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (460e48e9-0b00-4cc7-a0ef-b2060d0bc512) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:53.279413 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (460e48e9-0b00-4cc7-a0ef-b2060d0bc512) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:26:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-358198478',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=ecc91c7d-b5d2-4c67-b67f-7ab4a1c810c6,host='ncdv-master-1@ceph#ceph',id=396870ce-d1dc-4fee-9a2c-fde9c607bc20,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='aaed16ff29234c81b161434120a4bf12',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:26:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:26:52Z,use_quota=True,user_id='41c3af0358084a9da8f56f0b37ed7834',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(7496bac0-cc47-4cbf-96fb-5e89149e8a02),volume_type_id=7496bac0-cc47-4cbf-96fb-5e89149e8a02)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:53.288242 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (53bd77af-d085-498c-a1ca-1c15c5dca4ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:53.295231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (53bd77af-d085-498c-a1ca-1c15c5dca4ab) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:53.301367 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3873f702-7a59-4b57-9b90-35652dc9ff11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:53.325571 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3873f702-7a59-4b57-9b90-35652dc9ff11) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '396870ce-d1dc-4fee-9a2c-fde9c607bc20', 'volume_name': 'volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:53.338437 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa96cf47-5570-4b07-85e9-edc929804fab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:53.347715 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa96cf47-5570-4b07-85e9-edc929804fab) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:53.360411 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f5967c09-4709-4a2d-8f78-f5e5a17ac51b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:53.367402 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Volume 396870ce-d1dc-4fee-9a2c-fde9c607bc20: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20', 'volume_size': 1} May 28 21:26:53.367796 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] creating volume 'volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:26:53.368851 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:53.458401 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] Created reservations ['d47f6f5c-7cbc-4e16-99fd-c011acb101e7', 'dc81e6e5-0214-4c48-8d68-307fd0662053', '87fd7397-9743-41d2-942b-066bb71d275a', 'd165cd5e-0666-4311-933c-3609da277328'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:26:53.594286 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=56,cinder:UPDATE=36,cinder:INSERT=8 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:26:53.732540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:53.804321 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:26:53.883957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:26:53.895564 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] Deleted volume successfully. May 28 21:26:54.039802 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f37e3c35-5c3c-4329-88fb-5916061121ff tempest-VolumesCloneTest-1257877467 None] Lock "b'cinder-f21114e7-185b-4f50-9177-b5bdc86ab943-delete_volume'" released by "delete_volume" :: held 7.289s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:26:54.089174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:26:54.091865 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:26:54.104786 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f5967c09-4709-4a2d-8f78-f5e5a17ac51b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:54.104786 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8024d28d-f2ef-491d-b794-20021a319ea5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:26:54.166587 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20 (396870ce-d1dc-4fee-9a2c-fde9c607bc20): created successfully May 28 21:26:54.170261 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8024d28d-f2ef-491d-b794-20021a319ea5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:26:54.177346 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (e1c6f995-5114-4a3b-9291-a7fef67b6914) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:26:54.235486 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-316cb2ae-fffb-490e-a051-c8d05d52caf5 tempest-GroupSnapshotsTest-1289745910 None] Created volume successfully. May 28 21:26:55.018244 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-68b9b8a1-9d8f-4881-bf4b-a5f9c1c5f4a0 tempest-GroupSnapshotsTest-1289745910 None] Update group completed successfully. May 28 21:26:56.365860 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-718db990-6612-4582-a9da-778e0c390ad2 tempest-GroupSnapshotsTest-1289745910 None] Update group completed successfully. May 28 21:26:57.682022 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-cac2567d-73b8-4e23-92e0-e5b5d0576e68 tempest-GroupSnapshotsTest-1289745910 None] Update group completed successfully. May 28 21:26:58.919220 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c0272f59-3fa4-47df-93fb-6824024a5ab2 tempest-GroupSnapshotsTest-1289745910 None] Update group completed successfully. May 28 21:27:00.577945 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:00.626231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Flow 'volume_create_manager' (7adaf072-38cb-4e49-978d-ad22b0c936a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:00.637542 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1cce9bcb-483d-4f64-a7e7-92f70c3107f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:00.667812 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1cce9bcb-483d-4f64-a7e7-92f70c3107f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesNegativeTest-Volume-1052692047',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=0419d4b6-6d29-4a6b-8281-8edae07353bb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='760e1d0cdcff4948bc9f8849ecc56e1c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:00Z,use_quota=True,user_id='d470c6d9a88d4b758d9df497c450cffb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:00.680561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6b8e68aa-6e47-41b2-8efa-57eb2ba4d3be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:00.686541 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6b8e68aa-6e47-41b2-8efa-57eb2ba4d3be) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:00.707682 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a81272d7-746a-411d-b77c-058fa60d3ed8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:00.712651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a81272d7-746a-411d-b77c-058fa60d3ed8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '0419d4b6-6d29-4a6b-8281-8edae07353bb', 'volume_name': 'volume-0419d4b6-6d29-4a6b-8281-8edae07353bb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:00.720077 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f3147a4e-eb4b-4055-b28a-4e010e3d2e67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:00.725308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f3147a4e-eb4b-4055-b28a-4e010e3d2e67) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:00.731250 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4144b5f2-1ac0-4838-bd67-2ce8abd68f1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:00.734529 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Volume 0419d4b6-6d29-4a6b-8281-8edae07353bb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-0419d4b6-6d29-4a6b-8281-8edae07353bb', 'volume_size': 1} May 28 21:27:00.735200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] creating volume 'volume-0419d4b6-6d29-4a6b-8281-8edae07353bb' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:00.736017 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:00.949650 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:27:00.951044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:01.118637 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:01.176400 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:01.191138 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-93cc0eb8-33e1-4b29-b882-c25ed1be76fd tempest-GroupSnapshotsTest-1289745910 None] GroupSnapshot 28a6bd16-6327-438e-a7eb-699508fdece2: creating. May 28 21:27:01.241130 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93cc0eb8-33e1-4b29-b882-c25ed1be76fd tempest-GroupSnapshotsTest-1289745910 None] Group snapshot 28a6bd16-6327-438e-a7eb-699508fdece2: creating. {{(pid=135921) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4082}} May 28 21:27:01.292377 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93cc0eb8-33e1-4b29-b882-c25ed1be76fd tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:01.567806 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:01.571558 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:01.573653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (4144b5f2-1ac0-4838-bd67-2ce8abd68f1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0419d4b6-6d29-4a6b-8281-8edae07353bb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:01.579590 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fc32a6ad-1f48-425c-bcf1-1140abb7be13) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:01.670015 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Volume volume-0419d4b6-6d29-4a6b-8281-8edae07353bb (0419d4b6-6d29-4a6b-8281-8edae07353bb): created successfully May 28 21:27:01.680961 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (fc32a6ad-1f48-425c-bcf1-1140abb7be13) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:01.708794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Flow 'volume_create_manager' (7adaf072-38cb-4e49-978d-ad22b0c936a7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:02.267932 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c568e238-c465-455e-ad3e-5a7f2c9db652 tempest-VolumesNegativeTest-1222424159 None] Created volume successfully. May 28 21:27:03.425168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93cc0eb8-33e1-4b29-b882-c25ed1be76fd tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:06.326746 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-93cc0eb8-33e1-4b29-b882-c25ed1be76fd tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 28a6bd16-6327-438e-a7eb-699508fdece2: created successfully May 28 21:27:06.799919 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 28a6bd16-6327-438e-a7eb-699508fdece2: deleting May 28 21:27:06.817350 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 28a6bd16-6327-438e-a7eb-699508fdece2: deleting {{(pid=135921) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4257}} May 28 21:27:06.848274 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:09.653235 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:12.105784 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-1458451432 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-1458451432, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:12.107460 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-1458451432 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-1458451432, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:12.239868 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['ba084abd-a7df-42c9-9f69-f4cffee29b22', '24b2ba6b-433f-4080-a6a3-e33f872e5b78', 'f9535b81-596b-490f-a369-ec55d057ebbb', '8462b707-3648-4ecf-ab26-b2dda23c6513'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:12.436041 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=21,cinder:SELECT=74,cinder:DELETE=2,cinder:INSERT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:12.543263 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['db038c56-6f88-4a74-a5ff-aabbc461a041', 'a7ffb0e8-8a7d-4e04-ade3-73cc0a10cf1e', 'dc3418e1-34f9-42ce-b5ca-8a616b16469c', 'a6eeec92-70c6-4488-b39b-25d520aa2257'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:12.840027 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9df5ea93-52ce-4641-a052-df1760424486 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 28a6bd16-6327-438e-a7eb-699508fdece2: deleted successfully May 28 21:27:13.883534 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=68,cinder:UPDATE=26,cinder:DELETE=1,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:13.927637 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:14.080044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:14.080506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:14.183520 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] deleting rbd volume volume-396870ce-d1dc-4fee-9a2c-fde9c607bc20 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:17.079562 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a94f56ae-8984-4824-9e03-de9e39785e4e tempest-GroupsTest-1112740198 None] Group tempest-Group1-75880181: creating May 28 21:27:17.206495 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a94f56ae-8984-4824-9e03-de9e39785e4e tempest-GroupsTest-1112740198 None] Group tempest-Group1-75880181: created successfully May 28 21:27:17.226348 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a94f56ae-8984-4824-9e03-de9e39785e4e tempest-GroupsTest-1112740198 None] Create group completed successfully. May 28 21:27:17.811557 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:17.929813 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:17.933492 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:18.029041 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] deleting rbd volume volume-8c6b36aa-4f0f-44ae-839a-ca262e615ee0 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:18.655153 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ea1773b4-2f79-4a86-b853-3cc0cf9a3892 tempest-GroupsTest-1112740198 None] Group tempest-Group2-1572821314: creating May 28 21:27:18.757141 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ea1773b4-2f79-4a86-b853-3cc0cf9a3892 tempest-GroupsTest-1112740198 None] Group tempest-Group2-1572821314: created successfully May 28 21:27:18.770301 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ea1773b4-2f79-4a86-b853-3cc0cf9a3892 tempest-GroupsTest-1112740198 None] Create group completed successfully. May 28 21:27:20.812705 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:20.832120 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Flow 'volume_create_manager' (adef8af8-de47-45f8-ad2b-97352c36034c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:20.839427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ea660f9a-95fb-40a7-a280-9bd2cab14b63) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:20.851054 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['f60a4cb3-ae0c-4801-b7e6-6eb1d464d99c'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:20.857254 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ea660f9a-95fb-40a7-a280-9bd2cab14b63) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-volume-1692051153',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=24a1d113-e31e-4fb3-b733-df75b1e4cfa3,host='ncdv-master-1@ceph#ceph',id=18c4c42e-2708-4fc4-bd42-b9acfd33bd73,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9b13e9b82ca8467b8008de69cce86516',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:21Z,use_quota=True,user_id='7cc1828624064f028d1060ee393d2da3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(07da767c-5240-45d5-a5e4-f9d3745e7f3c),volume_type_id=07da767c-5240-45d5-a5e4-f9d3745e7f3c)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:20.862625 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03ac6ab7-dfb5-43b3-8514-ad42bbd824ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:20.866651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (03ac6ab7-dfb5-43b3-8514-ad42bbd824ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:20.871638 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (04cfe3ea-fab8-481a-bbdc-bfd054f2a582) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:20.875707 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (04cfe3ea-fab8-481a-bbdc-bfd054f2a582) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '18c4c42e-2708-4fc4-bd42-b9acfd33bd73', 'volume_name': 'volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:20.881024 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7fca11f1-bc0d-4465-b215-9ebc523c27d6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:20.884954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (7fca11f1-bc0d-4465-b215-9ebc523c27d6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:20.889517 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bc19fe8b-8c29-4875-a82c-f50609b18343) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:20.890702 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Volume 18c4c42e-2708-4fc4-bd42-b9acfd33bd73: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73', 'volume_size': 1} May 28 21:27:20.891081 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] creating volume 'volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:20.891896 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:20.932925 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-1458451432 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-1458451432, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:21.033994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['bcc2f695-74f3-41e8-a1d9-767706ff23a3', '45259f8e-11d3-44ba-ab3a-356211946a8e', '14a91760-e182-4d1d-82d9-ef7c52878b59', '7a2ac18d-97dc-4fec-a3d6-2b28d2406055'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:21.259796 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:21.260329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:21.264152 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bc19fe8b-8c29-4875-a82c-f50609b18343) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:21.281511 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d819f8be-b55c-4579-8d83-59b7cab373f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:21.371270 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Volume volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73 (18c4c42e-2708-4fc4-bd42-b9acfd33bd73): created successfully May 28 21:27:21.377656 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d819f8be-b55c-4579-8d83-59b7cab373f9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:21.389394 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Flow 'volume_create_manager' (adef8af8-de47-45f8-ad2b-97352c36034c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:21.399045 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['04483146-ce11-45f2-ad36-6c7f2fd98d84', '0b7a723b-10b3-4dcf-9453-7bbe01ab3ab3', '0ebe1892-93ef-4d2d-8d9c-325556ac166f', '47726969-fe50-43c3-aaaa-a684a3aa5cae'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:21.462633 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-168a3bd1-6edb-42ea-9f48-2b24c4ec3398 tempest-GroupsTest-1112740198 None] Created volume successfully. May 28 21:27:21.840231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:21.910322 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:21.950105 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:21.954513 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8e7134c2-29a8-4d2a-913e-dd3be09d29d9 tempest-GroupSnapshotsTest-1289745910 None] Delete group completed successfully. May 28 21:27:22.464788 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:22.571408 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:22.571838 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Volume volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:22.697263 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] deleting rbd volume volume-18c4c42e-2708-4fc4-bd42-b9acfd33bd73 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:23.530681 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ac7ea95b-1c97-4bd1-a9a6-990e10ea98d5 tempest-GroupSnapshotsTest-1289745910 None] Group tempest-GroupSnapshotsTest-Group-1688275720: creating May 28 21:27:23.621267 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ac7ea95b-1c97-4bd1-a9a6-990e10ea98d5 tempest-GroupSnapshotsTest-1289745910 None] Group tempest-GroupSnapshotsTest-Group-1688275720: created successfully May 28 21:27:23.630469 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ac7ea95b-1c97-4bd1-a9a6-990e10ea98d5 tempest-GroupSnapshotsTest-1289745910 None] Create group completed successfully. May 28 21:27:25.324957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Created reservations ['161670cf-af8d-4197-8292-5aa383836920'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:25.342312 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=66,cinder:UPDATE=29,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:25.346125 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-605255158 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-605255158, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:25.350971 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-605255158 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-605255158, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:25.473248 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Created reservations ['cc3039ba-581d-44ef-995a-ae34d86f2b65', '7f0c4112-606f-4ff1-87da-14de79b4062f', '0c7e7508-2200-49a2-9385-ed8ec0d76320', 'd3603f28-3a9b-45d9-baab-5d504ab5165e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:25.510190 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:25.546644 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (48bbbd2d-fd9e-4f18-a5fe-92e94b726be5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:25.561510 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (94e99915-f6bf-4de1-863c-7d6d042bc6de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:25.588260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (94e99915-f6bf-4de1-863c-7d6d042bc6de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:25Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsTest-Volume-147863290',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=4b16770f-8490-4706-af20-a4c442fb8199,host='ncdv-master-1@ceph#ceph',id=f1cc866f-44f6-4090-90e5-b21b25e547a8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='aaed16ff29234c81b161434120a4bf12',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:25Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:25Z,use_quota=True,user_id='41c3af0358084a9da8f56f0b37ed7834',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(6337f7e1-6e15-4605-907e-44c3d03c6c33),volume_type_id=6337f7e1-6e15-4605-907e-44c3d03c6c33)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:25.594777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2a333260-5a9b-4710-a1ed-94db55db948c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:25.600529 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2a333260-5a9b-4710-a1ed-94db55db948c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:25.605585 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6c593b93-9643-4c4e-a86c-ca9e4f6b62cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:25.609770 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6c593b93-9643-4c4e-a86c-ca9e4f6b62cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f1cc866f-44f6-4090-90e5-b21b25e547a8', 'volume_name': 'volume-f1cc866f-44f6-4090-90e5-b21b25e547a8', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:25.614894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (03299bf5-21c6-407b-9ccb-2370b7fd7f2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:25.619111 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (03299bf5-21c6-407b-9ccb-2370b7fd7f2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:25.623992 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9a800559-e869-491b-b0e8-8fa49f89f7d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:25.625160 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Volume f1cc866f-44f6-4090-90e5-b21b25e547a8: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f1cc866f-44f6-4090-90e5-b21b25e547a8', 'volume_size': 1} May 28 21:27:25.625534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] creating volume 'volume-f1cc866f-44f6-4090-90e5-b21b25e547a8' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:25.626914 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:25.677399 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=55,cinder:UPDATE=35,cinder:INSERT=9,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:25.965030 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:26.023412 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:26.053685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:26.073828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:26.074301 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:26.078560 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9a800559-e869-491b-b0e8-8fa49f89f7d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f1cc866f-44f6-4090-90e5-b21b25e547a8', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:26.083777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b83e675c-68f0-41b8-8ae8-48a2e71cc2ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:26.092642 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2ce1aeca-5221-44f2-a605-77155a5c4ba4 tempest-GroupsTest-1112740198 None] Delete group completed successfully. May 28 21:27:26.142606 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-f1cc866f-44f6-4090-90e5-b21b25e547a8 (f1cc866f-44f6-4090-90e5-b21b25e547a8): created successfully May 28 21:27:26.146501 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b83e675c-68f0-41b8-8ae8-48a2e71cc2ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:26.152096 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Flow 'volume_create_manager' (48bbbd2d-fd9e-4f18-a5fe-92e94b726be5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:26.228653 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c5814fc5-3669-47cf-9cca-a3bbff0d7479 tempest-GroupSnapshotsTest-1289745910 None] Created volume successfully. May 28 21:27:26.979412 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-49d48245-4bb7-441f-bf29-78016bd48b2a tempest-GroupSnapshotsTest-1289745910 None] GroupSnapshot 41bbf213-9d64-4490-92bf-92646dd421a8: creating. May 28 21:27:27.014646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-49d48245-4bb7-441f-bf29-78016bd48b2a tempest-GroupSnapshotsTest-1289745910 None] Group snapshot 41bbf213-9d64-4490-92bf-92646dd421a8: creating. {{(pid=135921) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4082}} May 28 21:27:27.041230 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-49d48245-4bb7-441f-bf29-78016bd48b2a tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:27.190391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e421ffcd-2044-4ed0-89bb-66def8c6e300 tempest-GroupsTest-1112740198 None] Created reservations ['df5d9866-4185-4591-84b9-277a350ecfed'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:27.384504 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e421ffcd-2044-4ed0-89bb-66def8c6e300 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:27.424189 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e421ffcd-2044-4ed0-89bb-66def8c6e300 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:27.473342 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-e421ffcd-2044-4ed0-89bb-66def8c6e300 tempest-GroupsTest-1112740198 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:27.479270 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e421ffcd-2044-4ed0-89bb-66def8c6e300 tempest-GroupsTest-1112740198 None] Delete group completed successfully. May 28 21:27:29.448971 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-72a2afac-31e6-4a4a-bacc-e06377d541bf tempest-GroupsTest-1112740198 None] Group tempest-GroupsTest-Group-7507286: creating May 28 21:27:29.534362 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-49d48245-4bb7-441f-bf29-78016bd48b2a tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 41bbf213-9d64-4490-92bf-92646dd421a8: created successfully May 28 21:27:29.639642 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-72a2afac-31e6-4a4a-bacc-e06377d541bf tempest-GroupsTest-1112740198 None] Group tempest-GroupsTest-Group-7507286: created successfully May 28 21:27:29.659143 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-72a2afac-31e6-4a4a-bacc-e06377d541bf tempest-GroupsTest-1112740198 None] Create group completed successfully. May 28 21:27:30.783159 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 41bbf213-9d64-4490-92bf-92646dd421a8: deleting May 28 21:27:30.804144 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 41bbf213-9d64-4490-92bf-92646dd421a8: deleting {{(pid=135921) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4257}} May 28 21:27:30.837075 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:31.366907 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] Lock "b'cinder-cf449889-2f06-41b0-9ec1-bd5ef48126ce-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:27:31.538719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:31.919545 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:31.924214 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] Volume volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:32.072111 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] deleting rbd volume volume-cf449889-2f06-41b0-9ec1-bd5ef48126ce {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:32.135698 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] Lock "b'cinder-0419d4b6-6d29-4a6b-8281-8edae07353bb-delete_volume'" acquired by "delete_volume" :: waited 0.043s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:27:32.193944 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:32.212781 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Flow 'volume_create_manager' (bc2b2f2a-440d-44fc-a197-e6873d11f135) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:32.219168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (802704b2-8ecc-42c3-a2f0-6139894ce6de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:32.235229 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (802704b2-8ecc-42c3-a2f0-6139894ce6de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-1098442853',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=ec4e5ba8-46cf-4432-ba0a-6f15794eb231,host='ncdv-master-1@ceph#ceph',id=d755936b-9b5f-464d-a8a3-18b3227c2e83,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9b13e9b82ca8467b8008de69cce86516',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:31Z,use_quota=True,user_id='7cc1828624064f028d1060ee393d2da3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e629d724-bb97-406f-a7e0-018548595ab0),volume_type_id=e629d724-bb97-406f-a7e0-018548595ab0)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:32.240639 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (933b7306-6a34-4631-8657-dcba7ae844f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:32.244684 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (933b7306-6a34-4631-8657-dcba7ae844f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:32.249680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (77818340-15d1-4c97-94d4-87ec0a2aea4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:32.253649 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (77818340-15d1-4c97-94d4-87ec0a2aea4c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd755936b-9b5f-464d-a8a3-18b3227c2e83', 'volume_name': 'volume-d755936b-9b5f-464d-a8a3-18b3227c2e83', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:32.258539 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a21b0f59-b134-4b39-a454-e29c53774b79) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:32.262361 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a21b0f59-b134-4b39-a454-e29c53774b79) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:32.266964 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e7fef8d5-8c64-49d8-b284-45a10b3b1f29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:32.268063 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Volume d755936b-9b5f-464d-a8a3-18b3227c2e83: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d755936b-9b5f-464d-a8a3-18b3227c2e83', 'volume_size': 1} May 28 21:27:32.268443 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] creating volume 'volume-d755936b-9b5f-464d-a8a3-18b3227c2e83' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:32.269072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:32.316696 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:32.495887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:32.496322 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] Volume volume-0419d4b6-6d29-4a6b-8281-8edae07353bb is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:32.805676 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] deleting rbd volume volume-0419d4b6-6d29-4a6b-8281-8edae07353bb {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:33.068170 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:33.068719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:33.072386 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e7fef8d5-8c64-49d8-b284-45a10b3b1f29) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d755936b-9b5f-464d-a8a3-18b3227c2e83', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:33.077375 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b7f661ce-2477-432d-8f51-9756c0770133) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:33.137995 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Volume volume-d755936b-9b5f-464d-a8a3-18b3227c2e83 (d755936b-9b5f-464d-a8a3-18b3227c2e83): created successfully May 28 21:27:33.141803 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b7f661ce-2477-432d-8f51-9756c0770133) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:33.148271 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Flow 'volume_create_manager' (bc2b2f2a-440d-44fc-a197-e6873d11f135) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:33.222462 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c762e70b-329f-49b2-9112-172089e6ae69 tempest-GroupsTest-1112740198 None] Created volume successfully. May 28 21:27:33.665189 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsTest-volume-type-878234487 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsTest-volume-type-878234487, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:33.665950 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsTest-volume-type-878234487 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsTest-volume-type-878234487, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:33.669666 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=17,cinder:SELECT=80,cinder:DELETE=2,cinder:INSERT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:33.742323 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['4cd7da2a-4dd8-43ce-ab69-92b09252dc7e', 'ba0544bc-3620-4708-9228-219f3ecbdf53', 'b5438d16-f7d6-478d-acf8-24aea05b8efc', 'b4f5a928-f900-4dee-bad3-3b6f101cd66c'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:34.079614 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e8fb2d70-a297-4497-845b-307c09cc30b9 tempest-GroupSnapshotsTest-1289745910 None] group_snapshot 41bbf213-9d64-4490-92bf-92646dd421a8: deleted successfully May 28 21:27:34.556946 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:34.608708 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Flow 'volume_create_manager' (80d9dbb2-454e-42cc-8639-7d68cbc5e497) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:34.625479 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (74583c27-78c6-425c-beb8-4164dc092479) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:34.655080 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (74583c27-78c6-425c-beb8-4164dc092479) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:34Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupsTest-Volume-82234943',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=ec4e5ba8-46cf-4432-ba0a-6f15794eb231,host='ncdv-master-1@ceph#ceph',id=1ea5c288-2880-481c-bf96-4e862d6e5ab6,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9b13e9b82ca8467b8008de69cce86516',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:34Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:34Z,use_quota=True,user_id='7cc1828624064f028d1060ee393d2da3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(e629d724-bb97-406f-a7e0-018548595ab0),volume_type_id=e629d724-bb97-406f-a7e0-018548595ab0)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:34.663285 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8918af5f-882f-4f4c-b642-8d04a8557b38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:34.670192 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8918af5f-882f-4f4c-b642-8d04a8557b38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:34.676628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ddc7501-5cfb-4f0e-9d54-46a2517f912f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:34.685952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5ddc7501-5cfb-4f0e-9d54-46a2517f912f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1ea5c288-2880-481c-bf96-4e862d6e5ab6', 'volume_name': 'volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:34.692607 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9320b7e7-7dd4-46e1-8a99-7cc76c6a5e33) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:34.697814 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9320b7e7-7dd4-46e1-8a99-7cc76c6a5e33) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:34.703824 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1730721f-0df0-49ac-a2bc-e20df6dafb0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:34.706763 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Volume 1ea5c288-2880-481c-bf96-4e862d6e5ab6: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6', 'volume_size': 1} May 28 21:27:34.706763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] creating volume 'volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:34.707153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:35.052669 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:35.285519 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:35.286690 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] Volume volume-f1cc866f-44f6-4090-90e5-b21b25e547a8 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:35.794743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] deleting rbd volume volume-f1cc866f-44f6-4090-90e5-b21b25e547a8 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:35.918828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:35.919139 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:35.923178 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (1730721f-0df0-49ac-a2bc-e20df6dafb0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:35.928709 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89a6e292-cde1-49e6-af3c-c22336fac9b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:35.977024 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Volume volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6 (1ea5c288-2880-481c-bf96-4e862d6e5ab6): created successfully May 28 21:27:35.980700 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89a6e292-cde1-49e6-af3c-c22336fac9b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:35.986289 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Flow 'volume_create_manager' (80d9dbb2-454e-42cc-8639-7d68cbc5e497) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:36.033697 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9c8b825b-f34f-470e-8e26-db5d2d841bd8 tempest-GroupsTest-1112740198 None] Created volume successfully. May 28 21:27:36.463463 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] Created reservations ['1ae4e41e-0d1a-4899-b1fc-536cbea7f052', '5a9db9c4-44ab-4498-b0c9-ef11b0862a1b', 'f0947a7d-10ab-4989-a06d-f98756184e5c', '77a3b662-4236-43f9-b92b-ce42e25f0457'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:36.694808 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:36.749107 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:36.778009 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:36.787790 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=62,cinder:UPDATE=31,cinder:INSERT=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:36.794268 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] Deleted volume successfully. May 28 21:27:36.899732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6abc895f-3b13-4eab-a6fd-1b1396fbf323 tempest-ServersNegativeTestJSON-498936791 None] Lock "b'cinder-cf449889-2f06-41b0-9ec1-bd5ef48126ce-delete_volume'" released by "delete_volume" :: held 5.533s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:27:36.947438 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-4fbaa9a2-c2af-4bd4-a48d-77c5008cb6b2 tempest-GroupsTest-1112740198 None] Update group completed successfully. May 28 21:27:36.968514 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] Lock "b'cinder-4ce735b7-0e4f-41cb-917f-48baf2562f1c-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:27:37.078940 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:37.256302 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:37.258294 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] Volume volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:37.393867 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] deleting rbd volume volume-4ce735b7-0e4f-41cb-917f-48baf2562f1c {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:37.507985 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] Created reservations ['774d3482-826b-41af-9935-6dd13048b5df', '6b9f2b63-3a16-4dfe-9fec-4df4a532ee62', 'a79a5b00-e91f-4918-b865-7ceabf0a0535', 'b41f06f1-5e12-4471-af27-0dac72262abf'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:37.725532 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:37.752576 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:37.795864 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:37.801617 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] Deleted volume successfully. May 28 21:27:37.892474 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8f06c655-8019-47fd-a48e-65c2bcf90205 tempest-VolumesNegativeTest-1222424159 None] Lock "b'cinder-0419d4b6-6d29-4a6b-8281-8edae07353bb-delete_volume'" released by "delete_volume" :: held 5.756s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:27:38.273957 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a94fb5cf-c91e-442c-99b9-05fec039a749 tempest-GroupsTest-1112740198 None] Update group completed successfully. May 28 21:27:39.754871 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:39.993828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:39.994270 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Volume volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:40.289354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] deleting rbd volume volume-1ea5c288-2880-481c-bf96-4e862d6e5ab6 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:40.303550 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=64,cinder:UPDATE=28,cinder:DELETE=3,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:40.377249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['afd07f0d-ea0a-42a7-9ea6-8a75f98ac690'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:40.432090 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsTest-volume-type-878234487 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsTest-volume-type-878234487, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:40.571082 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] Created reservations ['a60d4697-cb1d-4227-b588-b0dd90f1cc63', 'a7eeb31b-1c58-49bd-b476-507dd21d6b9c', 'c0b3afe5-49d3-461d-ba4f-ffc3085c967c', '3c8e0a91-14ca-4ade-aa9e-43cd7ed1952a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:41.160155 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:41.192609 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:41.280502 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:41.287745 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-bd6d91c4-8220-44ae-88fd-6e26c2837618 tempest-GroupSnapshotsTest-1289745910 None] Delete group completed successfully. May 28 21:27:41.571589 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] Created reservations ['bab1d456-fbae-4f7a-9789-1564a58db020', '319309a7-d4ad-4b7c-8c3f-39b6625d8971', 'ae3a7fc7-cb12-433a-adff-aa43287f1347', 'b7017d8e-dbdc-40c1-a9e9-fe825b85c550'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:41.799626 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:41.839073 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:41.953653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:41.967155 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] Deleted volume successfully. May 28 21:27:42.056932 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d667ea0c-76ee-4a04-85c5-b033c422664c tempest-ServersNegativeTestJSON-498936791 None] Lock "b'cinder-4ce735b7-0e4f-41cb-917f-48baf2562f1c-delete_volume'" released by "delete_volume" :: held 5.088s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:27:42.070377 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:42.101131 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Flow 'volume_create_manager' (21634a27-360b-4b1a-b054-938b06b3eded) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:42.108468 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1f65ba59-518c-4d0c-919c-b48f2a6c63da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:42.131456 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1f65ba59-518c-4d0c-919c-b48f2a6c63da) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:41Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotNegativeTestJSON-Volume-1926290665',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bfe62f99-826f-4080-aad9-acf882af0146,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='bcf76791bd3e47b68ca0381e604c3c53',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:42Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:42Z,use_quota=True,user_id='aa0d39e663594fb596b7be9def994e22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:42.134474 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ae2973b4-d661-4884-966a-bf9b8149cfce) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:42.139214 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ae2973b4-d661-4884-966a-bf9b8149cfce) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:42.145991 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5623f50a-c9b0-4a15-9dae-1006749044e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:42.150695 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5623f50a-c9b0-4a15-9dae-1006749044e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bfe62f99-826f-4080-aad9-acf882af0146', 'volume_name': 'volume-bfe62f99-826f-4080-aad9-acf882af0146', 'volume_size': 2}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:42.156449 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (03e94e93-bb41-48e6-b4f1-ddb3174611f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:42.160622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (03e94e93-bb41-48e6-b4f1-ddb3174611f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:42.165525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6c920a8e-9628-40e0-996e-76e2f80dff40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:42.166718 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Volume bfe62f99-826f-4080-aad9-acf882af0146: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bfe62f99-826f-4080-aad9-acf882af0146', 'volume_size': 2} May 28 21:27:42.167191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] creating volume 'volume-bfe62f99-826f-4080-aad9-acf882af0146' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:42.168135 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:43.084784 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:43.085446 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:43.089855 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6c920a8e-9628-40e0-996e-76e2f80dff40) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bfe62f99-826f-4080-aad9-acf882af0146', 'volume_size': 2}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:43.114000 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dad66f50-978a-4ba2-af54-edff72921ce3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:43.251029 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Volume volume-bfe62f99-826f-4080-aad9-acf882af0146 (bfe62f99-826f-4080-aad9-acf882af0146): created successfully May 28 21:27:43.259278 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dad66f50-978a-4ba2-af54-edff72921ce3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:43.267571 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Flow 'volume_create_manager' (21634a27-360b-4b1a-b054-938b06b3eded) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:43.287788 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=52,cinder:UPDATE=38,cinder:INSERT=9,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:43.386043 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dfedffdb-db5c-45fc-8b78-7a1d985b102b tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Created volume successfully. May 28 21:27:44.259169 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:44.416377 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:44.416897 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Volume volume-d755936b-9b5f-464d-a8a3-18b3227c2e83 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:44.514203 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] deleting rbd volume volume-d755936b-9b5f-464d-a8a3-18b3227c2e83 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:44.811166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-82340820-98c3-4be0-8ffc-f92f6aebaf98 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:47.358411 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-82340820-98c3-4be0-8ffc-f92f6aebaf98 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Create snapshot completed successfully May 28 21:27:48.165870 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-62637e6a-13ca-4e49-8efd-163d4b6d624a tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Lock "b'cinder-5813c0d8-8a90-4e9f-a257-e0308377ea38-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.031s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:27:48.170725 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-62637e6a-13ca-4e49-8efd-163d4b6d624a tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:48.689735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Created reservations ['776c8493-fc85-4419-be11-8d26efe93918'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:48.739538 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Deprecated: Default quota for resource: volumes_tempest-GroupsTest-volume-type-921096310 is set by the default quota flag: quota_volumes_tempest-GroupsTest-volume-type-921096310, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:48.740558 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupsTest-volume-type-921096310 is set by the default quota flag: quota_gigabytes_tempest-GroupsTest-volume-type-921096310, it is now deprecated. Please use the default quota class for default quota. May 28 21:27:48.828141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Created reservations ['dde384bf-14a2-4437-ad05-7f8490b99052', 'dafff424-2833-425e-b8e3-c1f123773694', 'e4770262-3b7c-43d6-beae-a6fbe318c953', '7f6e4cb5-1ad2-42d8-a17e-a08ef58b56b3'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:49.211659 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Created reservations ['f32e59e4-4871-4775-8665-ccdb010dbe08', '330c86cd-9d9f-47b5-abe5-efc6f59d0d96', '50b3ae3f-74c7-477f-a654-45ea0b587e57', '23874e73-aa75-41bb-b631-9cf96cbcbbd1'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:49.712992 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:49.775996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:49.839737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:49.843782 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-4ff9a7fd-d5f9-4522-b2c1-37df1959d8c4 tempest-GroupsTest-1112740198 None] Delete group completed successfully. May 28 21:27:50.015594 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-52ee39dd-c792-4126-ad4f-8097ba0ff02c tempest-GroupSnapshotsV319Test-1913887643 None] Group tempest-GroupSnapshotsV319Test-Group-358045433: creating May 28 21:27:50.051998 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-62637e6a-13ca-4e49-8efd-163d4b6d624a tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Created reservations ['20d87881-4e17-4855-8264-c762981a60dc', '54e720b3-fa3b-4bdf-bff9-36b4e6c4262d', '44612cf0-b36c-461a-b8b6-7bb14c267a03', '237f7030-69fb-4ea3-a0da-132187840341'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:50.118444 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-52ee39dd-c792-4126-ad4f-8097ba0ff02c tempest-GroupSnapshotsV319Test-1913887643 None] Group tempest-GroupSnapshotsV319Test-Group-358045433: created successfully May 28 21:27:50.128680 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-52ee39dd-c792-4126-ad4f-8097ba0ff02c tempest-GroupSnapshotsV319Test-1913887643 None] Create group completed successfully. May 28 21:27:50.363869 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-62637e6a-13ca-4e49-8efd-163d4b6d624a tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Delete snapshot completed successfully. May 28 21:27:50.369081 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-62637e6a-13ca-4e49-8efd-163d4b6d624a tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Lock "b'cinder-5813c0d8-8a90-4e9f-a257-e0308377ea38-delete_snapshot'" released by "delete_snapshot" :: held 2.203s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:27:50.580098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Lock "b'cinder-bfe62f99-826f-4080-aad9-acf882af0146-delete_volume'" acquired by "delete_volume" :: waited 0.034s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:27:50.734660 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:50.896815 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:27:50.901178 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Volume volume-bfe62f99-826f-4080-aad9-acf882af0146 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:27:50.984111 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] deleting rbd volume volume-bfe62f99-826f-4080-aad9-acf882af0146 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:27:52.551924 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:27:52.577525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Flow 'volume_create_manager' (33af26f3-9763-4993-9c59-29ade15585d3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:52.586462 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fd8529ed-3ef4-433e-9263-fd3984da961a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:52.603521 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fd8529ed-3ef4-433e-9263-fd3984da961a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:27:51Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsV319Test-Volume-824556631',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=c66ca159-2689-4b22-a506-56cbec94b86f,host='ncdv-master-1@ceph#ceph',id=51eeebac-2cd7-463b-ae51-e8aed445e5de,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7f6b1c0445ee4734bd092870f7ba1d4c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:27:52Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:27:52Z,use_quota=True,user_id='fbd2e63ff9c04ae38831b88193a574b2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(a9d8aea4-2fcf-4eb8-a270-f213956cb31d),volume_type_id=a9d8aea4-2fcf-4eb8-a270-f213956cb31d)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:52.609024 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ebd1638-bb0d-4915-991e-66b382b32e5e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:52.613470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7ebd1638-bb0d-4915-991e-66b382b32e5e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:52.619685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb3e55ef-04fe-4f34-a4c9-59af7069634e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:52.624599 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb3e55ef-04fe-4f34-a4c9-59af7069634e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '51eeebac-2cd7-463b-ae51-e8aed445e5de', 'volume_name': 'volume-51eeebac-2cd7-463b-ae51-e8aed445e5de', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:52.632500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (63910e6a-cd7b-4878-a24d-1a3f8c0f023c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:52.640499 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (63910e6a-cd7b-4878-a24d-1a3f8c0f023c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:52.646126 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c91f7033-a740-4e5b-bced-42fd0031e34c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:52.650771 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Volume 51eeebac-2cd7-463b-ae51-e8aed445e5de: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-51eeebac-2cd7-463b-ae51-e8aed445e5de', 'volume_size': 1} May 28 21:27:52.650771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] creating volume 'volume-51eeebac-2cd7-463b-ae51-e8aed445e5de' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:27:52.650771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:53.363807 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] extra_specs: {} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:27:53.364470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] extra_specs: {} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:27:53.373411 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c91f7033-a740-4e5b-bced-42fd0031e34c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-51eeebac-2cd7-463b-ae51-e8aed445e5de', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:53.381047 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f05e3ee3-1004-43ee-b3a4-198f164aa114) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:27:53.491595 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Volume volume-51eeebac-2cd7-463b-ae51-e8aed445e5de (51eeebac-2cd7-463b-ae51-e8aed445e5de): created successfully May 28 21:27:53.496389 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f05e3ee3-1004-43ee-b3a4-198f164aa114) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:27:53.503476 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Flow 'volume_create_manager' (33af26f3-9763-4993-9c59-29ade15585d3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:27:53.565134 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-106abdb2-9417-4366-815b-6eed92aeaf5e tempest-GroupSnapshotsV319Test-1913887643 None] Created volume successfully. May 28 21:27:54.124583 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=58,cinder:DELETE=2,cinder:UPDATE=34,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:54.222771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Created reservations ['bbaaf4f4-4ea2-4312-95c4-f4305eda6521', 'e6fc649e-1065-4b82-a1fd-4f9e8526401b', '989502cc-c107-4f11-989c-c293acb43130', 'fcb5e707-cdc7-4cdd-b631-c2eda3515cba'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:27:54.359343 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=58,cinder:UPDATE=30,cinder:DELETE=1,cinder:INSERT=11 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:27:54.491723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:54.557234 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:54.644159 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:27:54.656556 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Deleted volume successfully. May 28 21:27:54.763038 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6b2dada9-f094-42de-a527-97b3a2f1b046 tempest-VolumesSnapshotNegativeTestJSON-1198304572 None] Lock "b'cinder-bfe62f99-826f-4080-aad9-acf882af0146-delete_volume'" released by "delete_volume" :: held 4.179s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:27:55.236494 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-33b35103-5746-4f17-a57b-3ba2b0d06685 tempest-GroupSnapshotsV319Test-1913887643 None] GroupSnapshot 08287201-77cc-485f-99ed-acfa21dd4466: creating. May 28 21:27:55.288549 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-33b35103-5746-4f17-a57b-3ba2b0d06685 tempest-GroupSnapshotsV319Test-1913887643 None] Group snapshot 08287201-77cc-485f-99ed-acfa21dd4466: creating. {{(pid=135921) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4082}} May 28 21:27:55.315839 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-33b35103-5746-4f17-a57b-3ba2b0d06685 tempest-GroupSnapshotsV319Test-1913887643 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:27:57.434084 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-33b35103-5746-4f17-a57b-3ba2b0d06685 tempest-GroupSnapshotsV319Test-1913887643 None] group_snapshot 08287201-77cc-485f-99ed-acfa21dd4466: created successfully May 28 21:27:58.383696 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] group_snapshot 08287201-77cc-485f-99ed-acfa21dd4466: deleting May 28 21:27:58.444329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] group_snapshot 08287201-77cc-485f-99ed-acfa21dd4466: deleting {{(pid=135921) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4257}} May 28 21:27:58.486226 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:00.560378 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsV319Test-volume-type-1777601879 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsV319Test-volume-type-1777601879, it is now deprecated. Please use the default quota class for default quota. May 28 21:28:00.568190 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsV319Test-volume-type-1777601879 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsV319Test-volume-type-1777601879, it is now deprecated. Please use the default quota class for default quota. May 28 21:28:00.718253 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] Created reservations ['7f120606-7588-4ac4-9ff7-174a34609c97', '57a43647-df61-4fa0-885d-61d49d21a40f', '19c315bc-a0f3-4a1b-85c9-b54f31f5fbf7', '4c9d583e-c126-48b9-8730-fa187a641fdf'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:00.948556 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:28:00.949500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:01.038149 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:01.113626 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:28:01.198931 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5822bdf3-4aae-4fd1-af96-5bce15fed667 tempest-GroupSnapshotsV319Test-1913887643 None] group_snapshot 08287201-77cc-485f-99ed-acfa21dd4466: deleted successfully May 28 21:28:02.511429 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:02.742157 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:28:02.747138 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] Volume volume-51eeebac-2cd7-463b-ae51-e8aed445e5de is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:28:02.836069 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] deleting rbd volume volume-51eeebac-2cd7-463b-ae51-e8aed445e5de {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:28:06.340491 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] Created reservations ['029bc67b-9cd2-4ff4-8c29-feea7e376f86'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:06.387306 ncdv-master-1 cinder-volume[135921]: WARNING cinder.quota [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsV319Test-volume-type-1777601879 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsV319Test-volume-type-1777601879, it is now deprecated. Please use the default quota class for default quota. May 28 21:28:06.562158 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] Created reservations ['8d317e25-d112-4603-b14a-e62ac346f3be', '17684a29-731b-43da-a53e-ff6ed8abb680', 'a6dac69a-cc6c-4cdb-bb83-f8ff9d92d6a2', '9eded6ec-ee3b-4b7a-98dd-071d2768f9ed'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:07.105688 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=63,cinder:UPDATE=30,cinder:DELETE=1,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:07.252019 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:07.350486 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:07.421371 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:28:07.426747 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-becf9efb-942b-4e72-91d7-31c1ea8dec9e tempest-GroupSnapshotsV319Test-1913887643 None] Delete group completed successfully. May 28 21:28:18.808875 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0e03cb4a-25e4-43a1-b6be-fe10c49ff464 tempest-GroupsV320Test-740479232 None] Group tempest-GroupsV320Test-Group-1678271268: creating May 28 21:28:18.909286 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0e03cb4a-25e4-43a1-b6be-fe10c49ff464 tempest-GroupsV320Test-740479232 None] Group tempest-GroupsV320Test-Group-1678271268: created successfully May 28 21:28:18.934130 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0e03cb4a-25e4-43a1-b6be-fe10c49ff464 tempest-GroupsV320Test-740479232 None] Create group completed successfully. May 28 21:28:19.423233 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:28:19.455749 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Flow 'volume_create_manager' (f61bc914-f99b-4584-bdd5-679b13d93587) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:19.464084 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2eb9420f-6e78-4f7d-abc6-99041ca276d7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:19.484948 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2eb9420f-6e78-4f7d-abc6-99041ca276d7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:28:19Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AbsoluteLimitsTests-Volume-1824145313',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f0a328f4-10a6-4cb5-bcc8-78863a365915,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='810549f80c104ef787180ca456d45c81',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:28:19Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:28:19Z,use_quota=True,user_id='503314f28b4045f0bbaae8cdbfbfc989',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:19.491802 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b862c369-a024-4914-ac12-449e51488b65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:19.498290 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b862c369-a024-4914-ac12-449e51488b65) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:19.504308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d401820a-4c7a-4082-a803-152c66ce1c64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:19.510773 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d401820a-4c7a-4082-a803-152c66ce1c64) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f0a328f4-10a6-4cb5-bcc8-78863a365915', 'volume_name': 'volume-f0a328f4-10a6-4cb5-bcc8-78863a365915', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:19.517513 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f1bdeadf-c8fc-41ff-a04d-a6e0bb5808f2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:19.521783 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (f1bdeadf-c8fc-41ff-a04d-a6e0bb5808f2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:19.528499 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8cd1dd23-2b3e-4be2-ab84-85491cb02b08) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:19.530084 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Volume f0a328f4-10a6-4cb5-bcc8-78863a365915: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f0a328f4-10a6-4cb5-bcc8-78863a365915', 'volume_size': 1} May 28 21:28:19.530683 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] creating volume 'volume-f0a328f4-10a6-4cb5-bcc8-78863a365915' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:28:19.531624 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:20.039446 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:28:20.040552 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:28:20.044912 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8cd1dd23-2b3e-4be2-ab84-85491cb02b08) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f0a328f4-10a6-4cb5-bcc8-78863a365915', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:20.054543 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (65943c8c-6543-4d39-86f0-7d6ddb0e1743) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:20.159003 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Volume volume-f0a328f4-10a6-4cb5-bcc8-78863a365915 (f0a328f4-10a6-4cb5-bcc8-78863a365915): created successfully May 28 21:28:20.164603 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (65943c8c-6543-4d39-86f0-7d6ddb0e1743) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:20.175397 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Flow 'volume_create_manager' (f61bc914-f99b-4584-bdd5-679b13d93587) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:20.294694 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-099a655f-74f5-4e02-989d-306f718fe054 tempest-AbsoluteLimitsTests-1778557836 None] Created volume successfully. May 28 21:28:20.620493 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] Lock "b'cinder-f0a328f4-10a6-4cb5-bcc8-78863a365915-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:28:20.748572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:20.911296 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:28:20.911296 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] Volume volume-f0a328f4-10a6-4cb5-bcc8-78863a365915 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:28:20.998860 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] deleting rbd volume volume-f0a328f4-10a6-4cb5-bcc8-78863a365915 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:28:21.194295 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-28863c63-47fd-4938-b88a-beeb6d5f616e tempest-GroupsV320Test-740479232 None] Created reservations ['9e87640c-a066-43e9-bdc0-8f2dae82baf8'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:21.382707 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-28863c63-47fd-4938-b88a-beeb6d5f616e tempest-GroupsV320Test-740479232 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:21.491065 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-28863c63-47fd-4938-b88a-beeb6d5f616e tempest-GroupsV320Test-740479232 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:21.548613 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-28863c63-47fd-4938-b88a-beeb6d5f616e tempest-GroupsV320Test-740479232 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:28:21.555374 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-28863c63-47fd-4938-b88a-beeb6d5f616e tempest-GroupsV320Test-740479232 None] Delete group completed successfully. May 28 21:28:24.124370 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] Created reservations ['f993e832-9c15-4862-b64d-a8be66fdef5e', '5a506d3b-ef68-4a6b-af37-7e729ed441d7', '76b12b43-d621-42bf-a464-aa7eafd5f31d', '84eaefa4-1980-4248-81c5-5cfd141984ec'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:24.242888 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=71,cinder:UPDATE=23,cinder:INSERT=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:24.340954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:24.417768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:24.475710 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:28:24.481470 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] Deleted volume successfully. May 28 21:28:24.557583 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ac697993-bf4e-49c7-8fe5-8f6ec569cfa3 tempest-AbsoluteLimitsTests-1778557836 None] Lock "b'cinder-f0a328f4-10a6-4cb5-bcc8-78863a365915-delete_volume'" released by "delete_volume" :: held 3.941s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:28:30.740689 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:28:30.802972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Flow 'volume_create_manager' (142f763e-2c2f-4fdc-abfd-d0cbbdc0aae2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:30.814763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3b61ca68-bb6e-44c9-9be0-8e41095a1982) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:30.887279 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3b61ca68-bb6e-44c9-9be0-8e41095a1982) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:28:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesDeleteCascade-Volume-1157667606',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=26fc338d-b43c-44ea-ab5a-4fab891fe0a4,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7b6c916fa5074323b5c9a3e959f2f1e7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:28:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:28:31Z,use_quota=True,user_id='a9ca0f87ca18428fbdc5ae0eb5329342',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:30.901789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ea3b6745-c94e-4e8b-8fe2-08bcf2ae090d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:30.911275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ea3b6745-c94e-4e8b-8fe2-08bcf2ae090d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:30.921605 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e08d1cfb-111e-49b4-81bd-3e831723ed95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:30.926751 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e08d1cfb-111e-49b4-81bd-3e831723ed95) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '26fc338d-b43c-44ea-ab5a-4fab891fe0a4', 'volume_name': 'volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:30.934254 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d952f89f-e955-4798-95dc-d8a1f620e18b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:30.939717 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d952f89f-e955-4798-95dc-d8a1f620e18b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:30.946632 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (394586c1-dff7-4819-8af9-64713af515c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:30.948033 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Volume 26fc338d-b43c-44ea-ab5a-4fab891fe0a4: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4', 'volume_size': 1} May 28 21:28:30.948623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] creating volume 'volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:28:30.949706 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:31.422604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:28:31.423280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:28:31.427666 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (394586c1-dff7-4819-8af9-64713af515c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:31.434249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0acf6d55-0269-4e7b-aced-13a2e10eb554) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:31.547355 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Volume volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4 (26fc338d-b43c-44ea-ab5a-4fab891fe0a4): created successfully May 28 21:28:31.555781 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0acf6d55-0269-4e7b-aced-13a2e10eb554) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:31.575251 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Flow 'volume_create_manager' (142f763e-2c2f-4fdc-abfd-d0cbbdc0aae2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:31.685262 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6cd6e8e9-a515-4c7a-b9d3-214305508db7 tempest-VolumesDeleteCascade-844014945 None] Created volume successfully. May 28 21:28:33.664386 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c0d72e1c-b93f-4562-b342-90a319362087 tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:36.044477 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c0d72e1c-b93f-4562-b342-90a319362087 tempest-VolumesDeleteCascade-844014945 None] Create snapshot completed successfully May 28 21:28:37.633987 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-744e9556-cce8-47bb-a5bf-30249944529e tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:39.798198 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-744e9556-cce8-47bb-a5bf-30249944529e tempest-VolumesDeleteCascade-844014945 None] Create snapshot completed successfully May 28 21:28:39.803508 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=45,cinder:UPDATE=26,cinder:INSERT=3,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:40.983828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Lock "b'cinder-26fc338d-b43c-44ea-ab5a-4fab891fe0a4-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:28:41.114413 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Performing cascade delete. {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/manager.py:968}} May 28 21:28:41.148202 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Lock "b'cinder-77f589a5-31c4-4cc3-99de-126e2a2d31e8-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:28:41.173293 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:44.239220 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Created reservations ['a5c2078b-7021-4339-82ce-005c300b2a56', 'c102832e-1c4a-41db-8c3c-416939dee3c3', 'd5e9a095-752b-40f9-ab21-8f8317437f85', 'b28904b5-3b5c-471e-926a-22a01b5996d8'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:44.298002 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:28:44.315433 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Flow 'volume_create_manager' (790b7170-0929-43d2-8213-c86e19cacf3f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:44.322214 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2774033d-1352-47b3-8d40-884b54eff5f8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:44.339967 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (2774033d-1352-47b3-8d40-884b54eff5f8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:28:43Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesServicesTestJSON-Volume-1972968138',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4ed2192b2a514d379a4d6292f49d5a52',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:28:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:28:44Z,use_quota=True,user_id='20c731d95198430486c6bf5b3345efc7',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:44.346682 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0788231-b91c-4131-8516-09647a4b5d0a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:44.352141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0788231-b91c-4131-8516-09647a4b5d0a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:44.357540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7f2f2b07-ed38-4d2f-bd84-480107782d0d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:44.362636 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7f2f2b07-ed38-4d2f-bd84-480107782d0d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c', 'volume_name': 'volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:44.367874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cf06c754-736d-46b5-ae9e-0fc45bb8a239) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:44.372045 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cf06c754-736d-46b5-ae9e-0fc45bb8a239) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:44.376689 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c6fcb596-ec23-4765-91e8-b554356bcceb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:44.377759 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Volume bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c', 'volume_size': 1} May 28 21:28:44.378411 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] creating volume 'volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:28:44.379425 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:44.523745 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Delete snapshot completed successfully. May 28 21:28:44.528623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Lock "b'cinder-77f589a5-31c4-4cc3-99de-126e2a2d31e8-delete_snapshot'" released by "delete_snapshot" :: held 3.380s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:28:44.540582 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Lock "b'cinder-8479052f-b02a-479c-a6bc-347a7856486e-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:28:44.553263 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:44.870954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:28:44.878069 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:28:44.878069 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c6fcb596-ec23-4765-91e8-b554356bcceb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:44.880208 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f745a13e-ba5c-4a6e-b7d9-ea6126819b7c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:44.996694 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Volume volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c (bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c): created successfully May 28 21:28:45.000679 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f745a13e-ba5c-4a6e-b7d9-ea6126819b7c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:45.007530 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Flow 'volume_create_manager' (790b7170-0929-43d2-8213-c86e19cacf3f) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:45.146610 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-250e88bc-46e8-40ba-a973-ff11d7530f69 tempest-VolumesServicesTestJSON-190800321 None] Created volume successfully. May 28 21:28:46.500426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] Lock "b'cinder-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c-delete_volume'" acquired by "delete_volume" :: waited 0.016s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:28:46.724585 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:46.997399 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:28:46.997910 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] Volume volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:28:47.094085 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] deleting rbd volume volume-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:28:47.688925 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Created reservations ['b39ea327-6acb-4074-b019-44200d8a0877', 'ba5a7034-63ec-45c6-b25a-008ce4615a9f', '9aae54f6-9c4c-4eaa-928f-9bd6f3618860', '07bbe229-7d97-4ddd-a330-3d65584e2b31'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:47.911155 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Delete snapshot completed successfully. May 28 21:28:47.917408 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Lock "b'cinder-8479052f-b02a-479c-a6bc-347a7856486e-delete_snapshot'" released by "delete_snapshot" :: held 3.377s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:28:47.918199 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Snapshots deleted, issuing volume delete {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/manager.py:983}} May 28 21:28:47.919209 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:48.037162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:28:48.037752 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Volume volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:28:48.151877 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] deleting rbd volume volume-26fc338d-b43c-44ea-ab5a-4fab891fe0a4 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:28:52.111850 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] Created reservations ['b81c8ec3-9c22-4118-b3b9-5c81bae47456', 'fc84b15d-005d-4a44-b30c-86afb2d7c812', 'f7960dc7-004c-479e-a7f9-183d47b3821e', '00f12ae4-713c-4e36-a3af-fea0bcb71522'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:52.197341 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=65,cinder:UPDATE=25,cinder:DELETE=2,cinder:INSERT=8 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:52.261898 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:52.287529 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:52.313364 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:28:52.319056 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] Deleted volume successfully. May 28 21:28:52.397795 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-72316b4d-0cba-4160-a181-15ee57a83649 tempest-VolumesServicesTestJSON-190800321 None] Lock "b'cinder-bfcbf863-985a-4de9-b79f-e9d5d9ac6d8c-delete_volume'" released by "delete_volume" :: held 5.904s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:28:52.466385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Created reservations ['eab9a112-5502-418c-a15b-96ccb0486efd', '9fe4f7b6-d763-4eca-ba6f-b1d0ac89d9e0', '00e8d7a0-ce1c-43e0-9c51-5fbe5c93d831', '8fa8a8e2-e8a1-4262-beda-d4ddc2dca768'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:28:52.712348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:52.746401 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:52.883672 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:28:52.889518 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Deleted volume successfully. May 28 21:28:52.892259 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=54,cinder:UPDATE=37,cinder:INSERT=8,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:28:52.969078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a7b8685b-3738-4db9-a4d9-b87a422ce2bd tempest-VolumesDeleteCascade-844014945 None] Lock "b'cinder-26fc338d-b43c-44ea-ab5a-4fab891fe0a4-delete_volume'" released by "delete_volume" :: held 11.985s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:28:57.017788 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:28:57.039392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Flow 'volume_create_manager' (531a645c-29a6-4dc9-996e-42d089060d38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:57.053880 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5710414f-7bf7-4f91-9b8b-d0a102b3cdc6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:57.071078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5710414f-7bf7-4f91-9b8b-d0a102b3cdc6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:28:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-1245502555',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f64c199d-aafe-4b81-8b40-af7ac5fa31b5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='071f3ee8ccba4f4a891cac48aaa54d0a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:28:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:28:57Z,use_quota=True,user_id='38d2c184e2ad446cb0481b1421439771',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:57.076500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (83c4bc7f-26d6-44db-9732-ed98a3767b64) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:57.080952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (83c4bc7f-26d6-44db-9732-ed98a3767b64) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:57.087026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ee490992-7275-4be4-8a39-cdb5b1131674) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:57.091337 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ee490992-7275-4be4-8a39-cdb5b1131674) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'volume_name': 'volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:57.096567 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0cc377d9-29f1-4c9c-931a-2ac4103e4075) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:57.101172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0cc377d9-29f1-4c9c-931a-2ac4103e4075) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:57.105994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8ba1b5e-2494-4229-9285-157d3dd30792) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:57.107087 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Volume f64c199d-aafe-4b81-8b40-af7ac5fa31b5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'volume_size': 1} May 28 21:28:57.107675 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] creating volume 'volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:28:57.108441 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:28:57.902720 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:28:57.903103 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:28:57.918219 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8ba1b5e-2494-4229-9285-157d3dd30792) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:57.923660 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (da75acfe-3b07-402e-a8f6-831b355e4b46) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:28:57.998947 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Volume volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5 (f64c199d-aafe-4b81-8b40-af7ac5fa31b5): created successfully May 28 21:28:58.002755 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (da75acfe-3b07-402e-a8f6-831b355e4b46) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:28:58.030956 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Flow 'volume_create_manager' (531a645c-29a6-4dc9-996e-42d089060d38) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:28:58.094779 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-490d775a-23cb-4514-a492-473c553e977c tempest-VolumesBackupsTest-1716294911 None] Created volume successfully. May 28 21:29:00.949075 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:29:00.949780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:00.986630 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:01.036366 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:29:08.101434 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:DELETE=2,cinder:SELECT=4,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:14.077730 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-88393c47-045d-4a29-816a-f34580c4bc63 req-6ebc3521-fd2e-4105-ba23-dc60ce10dac9 tempest-VolumesBackupsTest-1716294911 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:29:15.296337 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-88393c47-045d-4a29-816a-f34580c4bc63 req-6ebc3521-fd2e-4105-ba23-dc60ce10dac9 tempest-VolumesBackupsTest-1716294911 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.217s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:29:15.300324 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-88393c47-045d-4a29-816a-f34580c4bc63 req-6ebc3521-fd2e-4105-ba23-dc60ce10dac9 tempest-VolumesBackupsTest-1716294911 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:29:15.425850 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-88393c47-045d-4a29-816a-f34580c4bc63 req-6ebc3521-fd2e-4105-ba23-dc60ce10dac9 tempest-VolumesBackupsTest-1716294911 None] Connection info returned from driver {'name': 'volumes/volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3685fd41-a8ad-4d7d-9fa4-5a90d9b3fef5'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:29:15.620533 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-88393c47-045d-4a29-816a-f34580c4bc63 req-6ebc3521-fd2e-4105-ba23-dc60ce10dac9 tempest-VolumesBackupsTest-1716294911 None] attachment_update completed successfully. May 28 21:29:16.888688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:29:16.936677 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Flow 'volume_create_manager' (f3b8686c-e75d-40ad-91b1-c5967ccd1d48) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:29:16.943725 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8c0cffcf-8b30-4a22-92bf-9c7d9b6dff0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:16.985757 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8c0cffcf-8b30-4a22-92bf-9c7d9b6dff0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:29:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1780704976',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ed95b6dd-f6d9-4cde-a354-d6ae7327ae42,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e3cc8c5f2fe44b87bf5c05eadb62936c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:29:17Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:29:17Z,use_quota=True,user_id='93f78df0f96c47ac97a200a4442db6da',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:16.992369 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (897897e5-dd37-4942-9987-d4b7d7dc5032) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:16.997660 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (897897e5-dd37-4942-9987-d4b7d7dc5032) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:17.003387 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d8bad81-a876-4613-8509-a02f38d56fba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:17.007499 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d8bad81-a876-4613-8509-a02f38d56fba) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'volume_name': 'volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:17.013692 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (552b5427-661b-4e78-8574-057a7f8991bf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:17.017666 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (552b5427-661b-4e78-8574-057a7f8991bf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:17.023560 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (20900dc9-7eef-4c3c-89d5-32e0fe076030) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:17.026616 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Volume ed95b6dd-f6d9-4cde-a354-d6ae7327ae42: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'volume_size': 1} May 28 21:29:17.026616 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] creating volume 'volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:29:17.026616 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:17.648952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:29:17.650134 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:29:17.656730 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (20900dc9-7eef-4c3c-89d5-32e0fe076030) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:17.662573 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07291fff-6f2c-4757-88d1-194aaf6ab55d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:17.791147 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Volume volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42 (ed95b6dd-f6d9-4cde-a354-d6ae7327ae42): created successfully May 28 21:29:17.795892 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07291fff-6f2c-4757-88d1-194aaf6ab55d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:17.812532 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Flow 'volume_create_manager' (f3b8686c-e75d-40ad-91b1-c5967ccd1d48) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:29:17.905288 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fa83a9fe-43d3-4acf-bb17-82461a44cdc7 tempest-AttachVolumeTestJSON-1447462214 None] Created volume successfully. May 28 21:29:19.212625 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:29:18Z,data_timestamp=2022-05-28T21:29:18Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-1185412182',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=0048f84a-8eab-4495-a615-827ef86560e9,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='071f3ee8ccba4f4a891cac48aaa54d0a',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='38d2c184e2ad446cb0481b1421439771',volume_id=f64c199d-aafe-4b81-8b40-af7ac5fa31b5) May 28 21:29:19.214435 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] continue_backup in rpcapi backup_id 0048f84a-8eab-4495-a615-827ef86560e9 {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:29:19.488465 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:29:20.471573 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.983s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:29:20.473420 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'f64c199d-aafe-4b81-8b40-af7ac5fa31b5', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:29:20.496642 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] Initialize volume connection completed successfully. May 28 21:29:21.233017 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-90c46067-7d56-498d-a519-b1ed091bb0d0 req-80a6fe2f-5984-425e-85cf-2805352e1751 tempest-AttachVolumeTestJSON-1447462214 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:29:21.965093 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-90c46067-7d56-498d-a519-b1ed091bb0d0 req-80a6fe2f-5984-425e-85cf-2805352e1751 tempest-AttachVolumeTestJSON-1447462214 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.732s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:29:21.967112 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-90c46067-7d56-498d-a519-b1ed091bb0d0 req-80a6fe2f-5984-425e-85cf-2805352e1751 tempest-AttachVolumeTestJSON-1447462214 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:29:22.088578 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-90c46067-7d56-498d-a519-b1ed091bb0d0 req-80a6fe2f-5984-425e-85cf-2805352e1751 tempest-AttachVolumeTestJSON-1447462214 None] Connection info returned from driver {'name': 'volumes/volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ed95b6dd-f6d9-4cde-a354-d6ae7327ae42', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'fb9750a1-d827-47bc-9e1c-d7a768fef237'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:29:22.268443 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-90c46067-7d56-498d-a519-b1ed091bb0d0 req-80a6fe2f-5984-425e-85cf-2805352e1751 tempest-AttachVolumeTestJSON-1447462214 None] attachment_update completed successfully. May 28 21:29:24.363319 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] Terminate volume connection completed successfully. May 28 21:29:24.397357 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9197d28f-6186-41e5-be56-bf095d874c7a tempest-VolumesBackupsTest-1716294911 None] Remove volume export completed successfully. May 28 21:29:25.811178 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=47,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:32.814435 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-a0ab7c13-019a-4fd1-8e86-a2f21464972d req-7d1fb544-f4b2-452c-85d9-2f2ea69118fe tempest-VolumesBackupsTest-1716294911 None] Terminate volume connection completed successfully. May 28 21:29:32.815174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-a0ab7c13-019a-4fd1-8e86-a2f21464972d req-7d1fb544-f4b2-452c-85d9-2f2ea69118fe tempest-VolumesBackupsTest-1716294911 None] Deleting attachment 3685fd41-a8ad-4d7d-9fa4-5a90d9b3fef5. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:29:37.984026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-f64c199d-aafe-4b81-8b40-af7ac5fa31b5-delete_volume'" acquired by "delete_volume" :: waited 0.033s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:29:38.104598 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:40.457994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] Volume volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:29:40.785523 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] deleting rbd volume volume-f64c199d-aafe-4b81-8b40-af7ac5fa31b5 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:29:43.342892 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=43,cinder:UPDATE=8 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:29:43.503614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] Created reservations ['789c56a0-fff4-48a6-a76e-ba5a27eed915', 'f2033026-8534-4025-ae1f-62ef644f80e2', '304258e7-14fc-4e05-b882-39d8df55837f', '73ce0118-1d54-4c96-ac7a-8db286eb438e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:29:43.723178 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:43.790769 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:43.880468 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:29:43.888189 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] Deleted volume successfully. May 28 21:29:43.978737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-73bb4b04-5fa1-4795-b0df-aa157c430b0e tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-f64c199d-aafe-4b81-8b40-af7ac5fa31b5-delete_volume'" released by "delete_volume" :: held 5.992s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:29:46.290970 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:29:46.351538 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Flow 'volume_create_manager' (af0c0e6f-1dbc-4b4c-979e-a63b8d30db51) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:29:46.365214 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (586b99b7-1a77-4a89-9b62-158ccae8d836) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:46.410113 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (586b99b7-1a77-4a89-9b62-158ccae8d836) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:29:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Volume-81433117',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='071f3ee8ccba4f4a891cac48aaa54d0a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:29:46Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:29:46Z,use_quota=True,user_id='38d2c184e2ad446cb0481b1421439771',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:46.422935 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8a26709c-0a3f-4fa6-a3ad-8ba7caf32a47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:46.427374 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8a26709c-0a3f-4fa6-a3ad-8ba7caf32a47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:46.442472 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (609f32fc-93a0-4024-a4de-07d5ee86fc9c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:46.580679 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (609f32fc-93a0-4024-a4de-07d5ee86fc9c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd', 'volume_name': 'volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:46.587758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cb937cc8-004a-44c5-b18d-1d5145050d99) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:46.592121 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (cb937cc8-004a-44c5-b18d-1d5145050d99) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:46.603820 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0b01ec81-2981-42af-9bc3-01ac8c65296d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:46.604993 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Volume dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:29:46.605575 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Cloning dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:29:46.629695 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.023s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:29:46.712027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 29, 46, 634548)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:29:46.712930 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:29:46.713576 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:29:46.719918 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.090s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:29:46.721198 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:29:46.731970 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 29, 46, 726483)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:29:46.733496 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:29:46.734964 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:29:46.763059 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:29:47.024587 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] creating snapshot='volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:29:48.091874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd.clone_snap' to 'volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:29:49.462962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:29:49.467154 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:29:49.468147 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:29:49.772727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:29:49.774139 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Marking volume dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:29:49.842998 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:29:49.843500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Creating volume glance metadata for volume dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:29:50.074229 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0b01ec81-2981-42af-9bc3-01ac8c65296d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:50.087793 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (348db4e9-95aa-4f91-9cd0-c4e7e06cd14a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:29:50.190761 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Volume volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd (dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd): created successfully May 28 21:29:50.194555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (348db4e9-95aa-4f91-9cd0-c4e7e06cd14a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:29:50.203775 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Flow 'volume_create_manager' (af0c0e6f-1dbc-4b4c-979e-a63b8d30db51) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:29:50.281678 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f0e7daa4-9017-4ed2-9c5e-d44dc62c909d tempest-VolumesBackupsTest-1716294911 None] Created volume successfully. May 28 21:29:51.323664 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:29:50Z,data_timestamp=2022-05-28T21:29:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsTest-Backup-48790400',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=efc30e7d-29b7-4042-973b-169c70808f0d,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='071f3ee8ccba4f4a891cac48aaa54d0a',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='38d2c184e2ad446cb0481b1421439771',volume_id=dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd) May 28 21:29:51.325348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] continue_backup in rpcapi backup_id efc30e7d-29b7-4042-973b-169c70808f0d {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:29:51.528582 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:29:52.301521 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.772s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:29:52.302995 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:29:52.324679 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] Initialize volume connection completed successfully. May 28 21:30:00.947606 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:30:00.949780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:30:01.009408 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:30:01.078547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:30:05.810401 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=48,cinder:UPDATE=22,cinder:INSERT=10,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:15.812163 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=22,cinder:UPDATE=4,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:20.670289 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-9ba7c57b-3c78-4114-afdb-a3c424e8f5dc req-70a1c0c4-dc21-453e-a770-9466bcd6f4c2 tempest-AttachVolumeTestJSON-1447462214 None] Terminate volume connection completed successfully. May 28 21:30:20.675606 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9ba7c57b-3c78-4114-afdb-a3c424e8f5dc req-70a1c0c4-dc21-453e-a770-9466bcd6f4c2 tempest-AttachVolumeTestJSON-1447462214 None] Deleting attachment fb9750a1-d827-47bc-9e1c-d7a768fef237. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:30:42.400285 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:42.415660 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] Terminate volume connection completed successfully. May 28 21:30:42.435351 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b550fd32-75d3-4aa4-bb52-1dffdf4e3e18 tempest-VolumesBackupsTest-1716294911 None] Remove volume export completed successfully. May 28 21:30:44.083898 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:30:44.110649 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Flow 'volume_create_manager' (f76e5c34-602f-4496-8141-77d7e7f9a8fd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:30:44.135578 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d6f1a894-20ab-46b8-9ffc-849a1c80dfec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:30:44.157992 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d6f1a894-20ab-46b8-9ffc-849a1c80dfec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:30:43Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_efc30e7d-29b7-4042-973b-169c70808f0d',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6589398c-11d6-483a-b206-0202e0f288ce,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='071f3ee8ccba4f4a891cac48aaa54d0a',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:30:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:30:44Z,use_quota=True,user_id='38d2c184e2ad446cb0481b1421439771',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:30:44.164036 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (32dd4ac5-ef2a-4fb3-b640-b874a7ea8ee5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:30:44.168394 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (32dd4ac5-ef2a-4fb3-b640-b874a7ea8ee5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:30:44.173572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b1a92725-b8c1-42f3-8b76-25bb64bc77ae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:30:44.178288 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b1a92725-b8c1-42f3-8b76-25bb64bc77ae) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6589398c-11d6-483a-b206-0202e0f288ce', 'volume_name': 'volume-6589398c-11d6-483a-b206-0202e0f288ce', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:30:44.183312 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da1f504f-0aa3-458d-84c4-419cae329c6d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:30:44.187510 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (da1f504f-0aa3-458d-84c4-419cae329c6d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:30:44.192470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (52d6770d-b582-43b6-a3ae-8838bea235b0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:30:44.193561 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Volume 6589398c-11d6-483a-b206-0202e0f288ce: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6589398c-11d6-483a-b206-0202e0f288ce', 'volume_size': 1} May 28 21:30:44.193987 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] creating volume 'volume-6589398c-11d6-483a-b206-0202e0f288ce' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:30:44.194759 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:30:44.699554 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:30:44.700366 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:30:44.706553 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (52d6770d-b582-43b6-a3ae-8838bea235b0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6589398c-11d6-483a-b206-0202e0f288ce', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:30:44.711333 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d31720f7-7c0c-440a-a454-f7502ea2306e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:30:44.794619 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Volume volume-6589398c-11d6-483a-b206-0202e0f288ce (6589398c-11d6-483a-b206-0202e0f288ce): created successfully May 28 21:30:44.798235 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d31720f7-7c0c-440a-a454-f7502ea2306e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:30:44.806899 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Flow 'volume_create_manager' (f76e5c34-602f-4496-8141-77d7e7f9a8fd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:30:44.920450 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Created volume successfully. May 28 21:30:46.030991 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:30:46.794127 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.763s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:30:46.795395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6589398c-11d6-483a-b206-0202e0f288ce', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '6589398c-11d6-483a-b206-0202e0f288ce', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:30:46.809812 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=15,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:30:46.812607 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Initialize volume connection completed successfully. May 28 21:31:00.846201 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1cfe6c36-4740-4067-b52e-77e3195ad67d tempest-BackendsCapabilitiesAdminTestsJSON-1653257569 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:ed9cca23-4c42-4484-ab51-c6d5cb2a97f0:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=135921) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4737}} May 28 21:31:00.880449 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a4384ba3-7754-494b-87b5-76e01d2f070d tempest-BackendsCapabilitiesAdminTestsJSON-1653257569 None] Obtained capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:ed9cca23-4c42-4484-ab51-c6d5cb2a97f0:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=135921) get_capabilities /opt/stack/cinder/cinder/volume/manager.py:4737}} May 28 21:31:00.951799 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:31:00.952721 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:00.977466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:01.000576 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:31:15.799900 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:15.805249 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=15,cinder:UPDATE=2,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:22.618129 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:31:22.638768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Flow 'volume_create_manager' (f899ae4c-b226-406a-9405-6de9f871dc85) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:31:22.660094 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fef467aa-e017-4330-be0c-e307123abb6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:22.678574 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fef467aa-e017-4330-be0c-e307123abb6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:31:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumeQuotasNegativeTestJSON-Volume-317455274',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8cd923db-70fe-458e-9209-34f12ec5a0ed,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5a8892908ee44862a6f48555703188b6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:31:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:31:22Z,use_quota=True,user_id='ab1c8bf8b24c4b1083e1e26840065a7a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:22.685615 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f66e18dd-4925-4459-bd19-edb78646f7d0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:22.690076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f66e18dd-4925-4459-bd19-edb78646f7d0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:22.712046 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7665d4e7-1cbf-4e9b-aca0-bdd4fb31727a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:22.717244 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7665d4e7-1cbf-4e9b-aca0-bdd4fb31727a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8cd923db-70fe-458e-9209-34f12ec5a0ed', 'volume_name': 'volume-8cd923db-70fe-458e-9209-34f12ec5a0ed', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:22.733202 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (12e245e8-03aa-4c18-b42a-0afdc97c3795) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:22.733202 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (12e245e8-03aa-4c18-b42a-0afdc97c3795) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:22.736487 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5a7db95e-1ab9-4419-aad7-027059cee12d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:22.737576 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Volume 8cd923db-70fe-458e-9209-34f12ec5a0ed: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8cd923db-70fe-458e-9209-34f12ec5a0ed', 'volume_size': 1} May 28 21:31:22.738008 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] creating volume 'volume-8cd923db-70fe-458e-9209-34f12ec5a0ed' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:31:22.738664 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:23.045456 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:31:23.064220 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Flow 'volume_create_manager' (af693b27-a667-42a9-a351-f2217d751286) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:31:23.070951 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5d62b856-7281-47df-a67a-77e2904dfb14) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:23.088288 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5d62b856-7281-47df-a67a-77e2904dfb14) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:31:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBackupRestore-volume-1304112313',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=bac5fe86-e590-414b-bff3-906130d40c5b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5241f87816134389b836a8eae1d53bd3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:31:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:31:23Z,use_quota=True,user_id='c93401a279c9403ca62627e829082968',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:23.093789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0cd9c910-7911-4bc0-a6ba-c5802204d9c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:23.097941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0cd9c910-7911-4bc0-a6ba-c5802204d9c5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:23.102941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f8f4accd-22ba-41a8-9b26-a2790fb47461) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:23.239992 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f8f4accd-22ba-41a8-9b26-a2790fb47461) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'bac5fe86-e590-414b-bff3-906130d40c5b', 'volume_name': 'volume-bac5fe86-e590-414b-bff3-906130d40c5b', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:23.247707 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5dc3b422-20f4-4993-b5ad-7c31c9fd5006) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:23.258388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5dc3b422-20f4-4993-b5ad-7c31c9fd5006) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:23.283252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (44d8e9a4-beac-467e-936e-5b7aea2c23bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:23.285413 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Volume bac5fe86-e590-414b-bff3-906130d40c5b: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-bac5fe86-e590-414b-bff3-906130d40c5b', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:31:23.286807 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Cloning bac5fe86-e590-414b-bff3-906130d40c5b from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:31:23.300603 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:31:23.491402 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 31, 23, 306983)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:31:23.492236 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:31:23.492809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:31:23.499192 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.199s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:31:23.500314 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:31:23.663268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 31, 23, 504840)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:31:23.664569 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:31:23.665622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:31:23.698061 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:23.926672 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] creating snapshot='volume-bac5fe86-e590-414b-bff3-906130d40c5b.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:31:27.494831 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-bac5fe86-e590-414b-bff3-906130d40c5b.clone_snap' to 'volume-bac5fe86-e590-414b-bff3-906130d40c5b' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:31:27.516772 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:31:27.517350 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:31:27.524378 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (5a7db95e-1ab9-4419-aad7-027059cee12d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8cd923db-70fe-458e-9209-34f12ec5a0ed', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:27.531423 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b3259141-1ac1-4b32-b0e5-88c3ac07bce5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:27.644698 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Volume volume-8cd923db-70fe-458e-9209-34f12ec5a0ed (8cd923db-70fe-458e-9209-34f12ec5a0ed): created successfully May 28 21:31:27.648873 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b3259141-1ac1-4b32-b0e5-88c3ac07bce5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:27.664341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Flow 'volume_create_manager' (f899ae4c-b226-406a-9405-6de9f871dc85) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:31:27.956523 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-93d18e25-d38e-4b6a-8f26-68566fd61ddf tempest-VolumeQuotasNegativeTestJSON-485618627 None] Created volume successfully. May 28 21:31:30.306251 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] Lock "b'cinder-8cd923db-70fe-458e-9209-34f12ec5a0ed-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:31:30.403378 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:30.756397 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:31:30.756852 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] Volume volume-8cd923db-70fe-458e-9209-34f12ec5a0ed is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:31:31.046909 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] deleting rbd volume volume-8cd923db-70fe-458e-9209-34f12ec5a0ed {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:31:31.189303 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:31:31.190582 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:31:31.190943 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:31:32.391809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:31:32.395933 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Marking volume bac5fe86-e590-414b-bff3-906130d40c5b as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:31:32.483357 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to bac5fe86-e590-414b-bff3-906130d40c5b. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:31:32.483805 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Creating volume glance metadata for volume bac5fe86-e590-414b-bff3-906130d40c5b backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:31:32.741394 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (44d8e9a4-beac-467e-936e-5b7aea2c23bd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-bac5fe86-e590-414b-bff3-906130d40c5b', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:32.746024 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8495575b-53ac-421f-9dbd-caf4bf3a8304) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:32.856316 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Volume volume-bac5fe86-e590-414b-bff3-906130d40c5b (bac5fe86-e590-414b-bff3-906130d40c5b): created successfully May 28 21:31:32.859846 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (8495575b-53ac-421f-9dbd-caf4bf3a8304) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:32.865328 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Flow 'volume_create_manager' (af693b27-a667-42a9-a351-f2217d751286) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:31:32.918750 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-789ea10c-97a2-4c76-baa6-d8ea801b14c5 tempest-TestVolumeBackupRestore-1923413318 None] Created volume successfully. May 28 21:31:34.102112 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:31:33Z,data_timestamp=2022-05-28T21:31:33Z,deleted=False,deleted_at=None,display_description=None,display_name=None,encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=7039d769-0e89-476f-a644-872f3ecd222f,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='5241f87816134389b836a8eae1d53bd3',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='c93401a279c9403ca62627e829082968',volume_id=bac5fe86-e590-414b-bff3-906130d40c5b) May 28 21:31:34.102894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] continue_backup in rpcapi backup_id 7039d769-0e89-476f-a644-872f3ecd222f {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:31:34.251183 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:31:34.953127 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.702s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:31:34.954206 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-bac5fe86-e590-414b-bff3-906130d40c5b', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'bac5fe86-e590-414b-bff3-906130d40c5b', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:31:34.970550 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] Initialize volume connection completed successfully. May 28 21:31:37.243860 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] Created reservations ['79661115-fd3a-43e2-a009-5a7618c193f7', 'b9698102-348a-4d59-b3fb-b72d0a9c7a6b', '22318f97-0c22-4421-84d9-0ac8e02d4fec', '84c6ab3a-b26f-4f70-ab1e-9259e0afc2e3'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:31:37.428326 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:37.472614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:37.500438 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:31:37.509912 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] Deleted volume successfully. May 28 21:31:37.616755 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-004deb6e-8d55-4ac8-9c0f-c9662005d705 tempest-VolumeQuotasNegativeTestJSON-485618627 None] Lock "b'cinder-8cd923db-70fe-458e-9209-34f12ec5a0ed-delete_volume'" released by "delete_volume" :: held 7.310s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:31:40.373789 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Terminate volume connection completed successfully. May 28 21:31:40.390244 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-62cfa036-8152-4889-89d6-bab314a69fc2 tempest-VolumesBackupsTest-1716294911 None] Remove volume export completed successfully. May 28 21:31:41.212390 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-6589398c-11d6-483a-b206-0202e0f288ce-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:31:41.314993 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:45.740540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] Volume volume-6589398c-11d6-483a-b206-0202e0f288ce is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:31:47.011687 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] deleting rbd volume volume-6589398c-11d6-483a-b206-0202e0f288ce {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:31:47.131750 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:31:47.151417 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (4cb9831a-8b99-4b5c-b152-7fb78f55010c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:31:47.158072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6165827a-eea0-479f-b2cd-2ad92f7ca1b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:47.173494 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6165827a-eea0-479f-b2cd-2ad92f7ca1b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:31:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-491801320',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=b31650fd-bab6-499a-bca7-af6b551f165e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6ed82f578e214948a9311cbdbb55a027',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:31:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:31:45Z,use_quota=True,user_id='8d033e99f9294e348b01c59a970931d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:47.178739 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ca0ace76-cc5d-4385-8984-f9553099325d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:47.183377 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ca0ace76-cc5d-4385-8984-f9553099325d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:47.188602 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9ed7e282-c771-464e-8e64-cfb95333caa3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:47.192737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9ed7e282-c771-464e-8e64-cfb95333caa3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b31650fd-bab6-499a-bca7-af6b551f165e', 'volume_name': 'volume-b31650fd-bab6-499a-bca7-af6b551f165e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:47.198997 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4d6ea65d-47f7-422e-939b-f8a5f9978fbf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:47.202968 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4d6ea65d-47f7-422e-939b-f8a5f9978fbf) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:47.207651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fb6641ed-bcba-4877-9557-f4996e4fbd5b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:47.211247 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Volume b31650fd-bab6-499a-bca7-af6b551f165e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b31650fd-bab6-499a-bca7-af6b551f165e', 'volume_size': 1} May 28 21:31:47.211247 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] creating volume 'volume-b31650fd-bab6-499a-bca7-af6b551f165e' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:31:47.211247 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:48.527051 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:31:48.527712 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:31:48.531223 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fb6641ed-bcba-4877-9557-f4996e4fbd5b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b31650fd-bab6-499a-bca7-af6b551f165e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:48.536414 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aea7c123-63a3-40ee-8b24-820a79be72ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:31:48.604145 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-b31650fd-bab6-499a-bca7-af6b551f165e (b31650fd-bab6-499a-bca7-af6b551f165e): created successfully May 28 21:31:48.608957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (aea7c123-63a3-40ee-8b24-820a79be72ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:31:48.615043 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (4cb9831a-8b99-4b5c-b152-7fb78f55010c) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:31:48.751724 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6a215a1a-29de-43b6-81d7-5f8a6fa0e8ac tempest-VolumesSnapshotTestJSON-961770574 None] Created volume successfully. May 28 21:31:50.453133 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=56,cinder:UPDATE=15,cinder:INSERT=5,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:50.453876 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=48,cinder:UPDATE=18,cinder:DELETE=2,cinder:INSERT=11 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:31:50.531140 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-76ab3e76-dac4-40a6-bb23-539315ad2f34 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:56.283993 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-76ab3e76-dac4-40a6-bb23-539315ad2f34 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:31:57.573495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.driver [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Creating a new backup for volume volume-b31650fd-bab6-499a-bca7-af6b551f165e. {{(pid=135921) _get_backup_volume_temp_volume /opt/stack/cinder/cinder/volume/driver.py:1213}} May 28 21:31:57.719699 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] cloning volumes/volume-b31650fd-bab6-499a-bca7-af6b551f165e@snapshot-c08d393b-3af5-4bf1-a0d6-9899b9419fa0 to volume-efe7548f-0d36-4ab9-a529-aadaf622854f {{(pid=135921) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1098}} May 28 21:31:57.720603 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:57.775352 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:31:59.856665 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:31:59.857348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:00.187260 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:31:56Z,data_timestamp=2022-05-28T21:31:50Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-1184533321',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=ab1c0376-6cb5-410e-a6a5-39226d83b255,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='6ed82f578e214948a9311cbdbb55a027',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='c08d393b-3af5-4bf1-a0d6-9899b9419fa0',status='creating',temp_snapshot_id=None,temp_volume_id='efe7548f-0d36-4ab9-a529-aadaf622854f',updated_at=None,user_id='8d033e99f9294e348b01c59a970931d5',volume_id=b31650fd-bab6-499a-bca7-af6b551f165e) May 28 21:32:00.188065 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] continue_backup in rpcapi backup_id ab1c0376-6cb5-410e-a6a5-39226d83b255 {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:32:00.381298 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:32:00.956795 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:32:00.960743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:01.015476 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:01.055117 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:32:01.058901 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.683s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:32:01.059665 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-efe7548f-0d36-4ab9-a529-aadaf622854f', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'efe7548f-0d36-4ab9-a529-aadaf622854f', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:32:01.078405 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Initialize volume connection completed successfully. May 28 21:32:03.101166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] Created reservations ['e5a503ec-c030-4147-aa7a-79cf229de9a1', '3e4263d3-8162-4fe2-a2d3-a7f4c3d67660', '78bd2176-ba3d-4754-81a9-c9003d992f0d', '7ba11eb9-4a39-4810-ae45-fbcbb1776db6'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:32:03.330370 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:03.355605 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:03.381298 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:32:03.387659 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] Deleted volume successfully. May 28 21:32:03.476402 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-46e7d186-f505-4d34-afa4-28504f2c8d3f tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-6589398c-11d6-483a-b206-0202e0f288ce-delete_volume'" released by "delete_volume" :: held 22.264s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:32:13.925109 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:32:13.944372 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Flow 'volume_create_manager' (2a0047ce-ac1d-4bd0-bc57-4893689193ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:13.951915 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ad3ff307-744f-4454-bb01-219b6c82691c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:13.968518 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ad3ff307-744f-4454-bb01-219b6c82691c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:32:13Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-513873282',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=60d078cc-0a8b-44c0-924e-2d287ff7f3bd,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e3cc8c5f2fe44b87bf5c05eadb62936c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:32:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:32:14Z,use_quota=True,user_id='93f78df0f96c47ac97a200a4442db6da',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:13.973706 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (13d6c986-e78c-445d-a033-07e054ac85dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:13.977763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (13d6c986-e78c-445d-a033-07e054ac85dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:13.982813 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59cb5054-d66d-4da4-a5e2-9980bd1ed6b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:13.986825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (59cb5054-d66d-4da4-a5e2-9980bd1ed6b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'volume_name': 'volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:13.991671 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (35be4367-9214-41bc-958a-2814693dfda3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:13.996094 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (35be4367-9214-41bc-958a-2814693dfda3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:14.001097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f5634789-1fa2-4c4e-b89b-d75e6e86153b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:14.002229 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Volume 60d078cc-0a8b-44c0-924e-2d287ff7f3bd: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'volume_size': 1} May 28 21:32:14.002638 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] creating volume 'volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:32:14.003787 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:15.290411 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:32:15.291093 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:15.294561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f5634789-1fa2-4c4e-b89b-d75e6e86153b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:15.299282 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3248ef0-73a8-4d54-a8da-e08cf52400e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:15.409539 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Volume volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd (60d078cc-0a8b-44c0-924e-2d287ff7f3bd): created successfully May 28 21:32:15.413706 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3248ef0-73a8-4d54-a8da-e08cf52400e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:15.419587 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Flow 'volume_create_manager' (2a0047ce-ac1d-4bd0-bc57-4893689193ca) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:15.506512 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-632daa76-9f5c-4803-bcfe-a7473136a61a tempest-AttachVolumeTestJSON-1447462214 None] Created volume successfully. May 28 21:32:16.811456 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-21f332c3-c426-482d-b02a-48e9fd499e6f req-ff195bd4-8a91-4e45-af76-8beae450e331 tempest-AttachVolumeTestJSON-1447462214 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:32:17.466953 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-21f332c3-c426-482d-b02a-48e9fd499e6f req-ff195bd4-8a91-4e45-af76-8beae450e331 tempest-AttachVolumeTestJSON-1447462214 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.655s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:32:17.467849 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-21f332c3-c426-482d-b02a-48e9fd499e6f req-ff195bd4-8a91-4e45-af76-8beae450e331 tempest-AttachVolumeTestJSON-1447462214 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:32:17.553336 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-21f332c3-c426-482d-b02a-48e9fd499e6f req-ff195bd4-8a91-4e45-af76-8beae450e331 tempest-AttachVolumeTestJSON-1447462214 None] Connection info returned from driver {'name': 'volumes/volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '60d078cc-0a8b-44c0-924e-2d287ff7f3bd', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '97059e3c-0635-46e9-a572-c8b62ca39219'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:32:17.675333 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-21f332c3-c426-482d-b02a-48e9fd499e6f req-ff195bd4-8a91-4e45-af76-8beae450e331 tempest-AttachVolumeTestJSON-1447462214 None] attachment_update completed successfully. May 28 21:32:22.073663 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:32:22.094924 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Flow 'volume_create_manager' (8caf310d-1bd2-4566-abe2-e86b410e7ee3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:22.101431 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a72e5dfb-188a-470e-bf50-8dcd933c61ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:22.117880 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a72e5dfb-188a-470e-bf50-8dcd933c61ff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:32:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeTestJSON-volume-1333283222',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=be8b4c83-8b39-4532-a3ec-dc28247b1e69,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e3cc8c5f2fe44b87bf5c05eadb62936c',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:32:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:32:22Z,use_quota=True,user_id='93f78df0f96c47ac97a200a4442db6da',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:22.123788 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8284cf59-e135-479f-aea1-95e27651740f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:22.128018 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8284cf59-e135-479f-aea1-95e27651740f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:22.132937 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (89945ef3-0798-4533-b018-39bc73d1a8c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:22.137793 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (89945ef3-0798-4533-b018-39bc73d1a8c6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'volume_name': 'volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:22.142748 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d8d0d121-c5ac-47de-8632-7590115fb23f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:22.146685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d8d0d121-c5ac-47de-8632-7590115fb23f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:22.151953 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a4c3f348-b684-46c8-b3d9-9fbebc71443b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:22.153033 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Volume be8b4c83-8b39-4532-a3ec-dc28247b1e69: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'volume_size': 1} May 28 21:32:22.153450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] creating volume 'volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:32:22.154293 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:22.982822 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:32:22.983358 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:22.987305 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a4c3f348-b684-46c8-b3d9-9fbebc71443b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:22.991948 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (45729f44-c809-4d40-ae5d-d3bbc0352870) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:22.996878 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=47,cinder:UPDATE=12,cinder:DELETE=1,cinder:INSERT=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:22.997608 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=35,cinder:UPDATE=18,cinder:INSERT=2,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:23.072369 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Volume volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69 (be8b4c83-8b39-4532-a3ec-dc28247b1e69): created successfully May 28 21:32:23.076029 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (45729f44-c809-4d40-ae5d-d3bbc0352870) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:23.081468 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Flow 'volume_create_manager' (8caf310d-1bd2-4566-abe2-e86b410e7ee3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:23.180699 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b527ff6d-3ef0-4e8b-a6b6-f751d5b874e4 tempest-AttachVolumeTestJSON-1447462214 None] Created volume successfully. May 28 21:32:23.416754 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:32:23.505741 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:26.554006 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] deleting rbd volume volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:32:26.585103 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] Terminate volume connection completed successfully. May 28 21:32:26.603605 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-fafc9251-a378-4cc7-8cdc-38fe2e884e36 req-735dc4af-cd3f-49bd-8a4b-51edb373f11b tempest-AttachVolumeTestJSON-1447462214 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:32:26.634872 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-29aaa688-ade9-4bcb-96f9-0bbe475ff858 tempest-TestVolumeBackupRestore-1923413318 None] Remove volume export completed successfully. May 28 21:32:27.254225 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-fafc9251-a378-4cc7-8cdc-38fe2e884e36 req-735dc4af-cd3f-49bd-8a4b-51edb373f11b tempest-AttachVolumeTestJSON-1447462214 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.651s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:32:27.255023 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fafc9251-a378-4cc7-8cdc-38fe2e884e36 req-735dc4af-cd3f-49bd-8a4b-51edb373f11b tempest-AttachVolumeTestJSON-1447462214 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:32:27.455062 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-fafc9251-a378-4cc7-8cdc-38fe2e884e36 req-735dc4af-cd3f-49bd-8a4b-51edb373f11b tempest-AttachVolumeTestJSON-1447462214 None] Connection info returned from driver {'name': 'volumes/volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'be8b4c83-8b39-4532-a3ec-dc28247b1e69', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '29d92002-ac89-4be5-ac19-57270d3e0ccb'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:32:27.634942 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-fafc9251-a378-4cc7-8cdc-38fe2e884e36 req-735dc4af-cd3f-49bd-8a4b-51edb373f11b tempest-AttachVolumeTestJSON-1447462214 None] attachment_update completed successfully. May 28 21:32:28.348454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:32:28.368569 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Flow 'volume_create_manager' (b942d118-4809-4419-b446-77044a484fcb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:28.375612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e4275bee-5391-4966-8187-8fa50da742a7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:28.392782 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e4275bee-5391-4966-8187-8fa50da742a7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:32:28Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_7039d769-0e89-476f-a644-872f3ecd222f',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5241f87816134389b836a8eae1d53bd3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:32:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:32:28Z,use_quota=True,user_id='c93401a279c9403ca62627e829082968',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:28.399252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7759327b-6139-4942-9d59-83a1e0de7448) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:28.403261 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7759327b-6139-4942-9d59-83a1e0de7448) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:28.408538 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ecbc10c-6676-4e43-bd5d-5193166bb224) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:28.412994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ecbc10c-6676-4e43-bd5d-5193166bb224) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'volume_name': 'volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:28.417719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fc154b7d-202f-40fb-a0e6-aef3dbdbb920) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:28.421619 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fc154b7d-202f-40fb-a0e6-aef3dbdbb920) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:28.426559 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7f176f5b-95f2-41b2-8a7a-341c7ec0ff9a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:28.427623 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Volume 66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'volume_size': 1} May 28 21:32:28.428021 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] creating volume 'volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:32:28.431763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:29.371194 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:32:29.371859 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:29.375635 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7f176f5b-95f2-41b2-8a7a-341c7ec0ff9a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:29.380152 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e7d99282-4af6-44f5-8338-50012ba6046c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:29.671955 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Volume volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0 (66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0): created successfully May 28 21:32:29.676799 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e7d99282-4af6-44f5-8338-50012ba6046c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:29.681046 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Flow 'volume_create_manager' (b942d118-4809-4419-b446-77044a484fcb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:29.757709 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Created volume successfully. May 28 21:32:30.569376 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:32:31.302319 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.733s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:32:31.303164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:32:31.320686 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Initialize volume connection completed successfully. May 28 21:32:31.899507 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-e0561019-1157-44dd-863f-a418ba710aca req-4c03c38c-d7a0-4577-bea5-5e11e68511ee tempest-AttachVolumeTestJSON-1447462214 None] Terminate volume connection completed successfully. May 28 21:32:31.900151 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-e0561019-1157-44dd-863f-a418ba710aca req-4c03c38c-d7a0-4577-bea5-5e11e68511ee tempest-AttachVolumeTestJSON-1447462214 None] Deleting attachment 97059e3c-0635-46e9-a572-c8b62ca39219. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:32:32.227945 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:32:32.251718 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Flow 'volume_create_manager' (e4f93163-bf02-4e05-9eed-a9647639eb89) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:32.258451 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (51db9fdd-012c-48ee-ad59-539d6c22e112) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:32.277817 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (51db9fdd-012c-48ee-ad59-539d6c22e112) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:32:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTestJSON-Volume-2118771571',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=719cf254-e14a-4776-876d-e0177461f78b,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='0ce4c7a9c74d441581a634afac6b8f8d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:32:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:32:32Z,use_quota=True,user_id='7882449f34df41708755bf3f902cf043',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:32.283908 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6ab2c1bb-9637-478d-900a-311154c9a43d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:32.288186 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6ab2c1bb-9637-478d-900a-311154c9a43d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:32.293640 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db88346f-beba-4f20-8327-5d5c560c1907) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:32.298168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db88346f-beba-4f20-8327-5d5c560c1907) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '719cf254-e14a-4776-876d-e0177461f78b', 'volume_name': 'volume-719cf254-e14a-4776-876d-e0177461f78b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:32.303093 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a27140cf-7c69-4117-a9f6-9a32060f654c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:32.307014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a27140cf-7c69-4117-a9f6-9a32060f654c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:32.312018 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (49c133d2-aef4-41f8-8e8a-cdf77eaad46c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:32.312987 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Volume 719cf254-e14a-4776-876d-e0177461f78b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-719cf254-e14a-4776-876d-e0177461f78b', 'volume_size': 1} May 28 21:32:32.313552 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] creating volume 'volume-719cf254-e14a-4776-876d-e0177461f78b' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:32:32.314771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:35.349281 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:32:35.921038 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] deleting parent snapshot volume-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:32:42.545124 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=34,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:42.545864 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=27,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:32:42.549301 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:32:42.550345 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:42.554011 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (49c133d2-aef4-41f8-8e8a-cdf77eaad46c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-719cf254-e14a-4776-876d-e0177461f78b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:42.558604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (203e8da0-a33c-479c-9f79-7fd2f6ba22c7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:42.598300 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-e43f6edb-9f18-4187-af96-f8fd13678057 req-07af2c98-836b-430a-9879-03a70d18bc61 tempest-AttachVolumeTestJSON-1447462214 None] Terminate volume connection completed successfully. May 28 21:32:42.598300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-e43f6edb-9f18-4187-af96-f8fd13678057 req-07af2c98-836b-430a-9879-03a70d18bc61 tempest-AttachVolumeTestJSON-1447462214 None] Deleting attachment 29d92002-ac89-4be5-ac19-57270d3e0ccb. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:32:42.722620 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] Created reservations ['1ec01b99-8bf1-4d9e-ab7e-a9ef1eff5dff', 'fce9d523-e7c4-448b-911a-fcc9d38b34ce', 'c38005f9-c8c9-4740-bfcf-b1f5c7410ac8', '6adc7f87-ec18-40fc-8ffa-7a599147a468'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:32:42.728111 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Volume volume-719cf254-e14a-4776-876d-e0177461f78b (719cf254-e14a-4776-876d-e0177461f78b): created successfully May 28 21:32:42.737365 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (203e8da0-a33c-479c-9f79-7fd2f6ba22c7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:42.738072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Flow 'volume_create_manager' (e4f93163-bf02-4e05-9eed-a9647639eb89) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:42.840767 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-709eb976-f776-4d79-b55a-df7a4d0b6c47 req-f8d2823b-ec5e-4ad6-938c-61b93b590474 tempest-VolumesGetTestJSON-675753615 None] Created volume successfully. May 28 21:32:42.956923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:42.980397 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:43.005753 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:32:43.011152 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] Deleted volume successfully. May 28 21:32:43.104204 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a762e425-c889-4aac-9289-90203cd32305 tempest-VolumesBackupsTest-1716294911 None] Lock "b'cinder-dd38b1bb-0ed4-4cb0-b594-5f3aa84501bd-delete_volume'" released by "delete_volume" :: held 19.687s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:32:44.120996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] Lock "b'cinder-719cf254-e14a-4776-876d-e0177461f78b-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:32:44.236440 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:45.129734 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:32:45.133204 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] Volume volume-719cf254-e14a-4776-876d-e0177461f78b is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:32:45.541474 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] deleting rbd volume volume-719cf254-e14a-4776-876d-e0177461f78b {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:32:47.174526 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:32:47.196480 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (3df735d5-b0a3-4d3f-82ac-cf0c4f769637) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:47.203101 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6002bfeb-8907-4e73-97da-b49555835f07) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:47.223484 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6002bfeb-8907-4e73-97da-b49555835f07) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:32:46Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1389502029',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=c76ca520-9eba-450b-86ae-0323cf62d27e,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:32:47Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:32:47Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:47.228954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (81477b7f-6343-403e-9d93-71f6e0068ab5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:47.233275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (81477b7f-6343-403e-9d93-71f6e0068ab5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:47.239257 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3829355-6a33-498f-aabb-64e79aa66e90) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:47.243455 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d3829355-6a33-498f-aabb-64e79aa66e90) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c76ca520-9eba-450b-86ae-0323cf62d27e', 'volume_name': 'volume-c76ca520-9eba-450b-86ae-0323cf62d27e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:47.251653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b2dbbdeb-3583-469e-a4e3-8e4a63c27bbb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:47.256876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b2dbbdeb-3583-469e-a4e3-8e4a63c27bbb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:47.262128 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2bcbcf3e-240b-46f5-ab84-44b34fbce6ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:47.263228 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Volume c76ca520-9eba-450b-86ae-0323cf62d27e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c76ca520-9eba-450b-86ae-0323cf62d27e', 'volume_size': 1} May 28 21:32:47.263625 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] creating volume 'volume-c76ca520-9eba-450b-86ae-0323cf62d27e' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:32:47.264725 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:49.520799 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:32:49.521400 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:49.525791 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (2bcbcf3e-240b-46f5-ab84-44b34fbce6ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c76ca520-9eba-450b-86ae-0323cf62d27e', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:49.530570 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dee969e3-3285-4377-8912-d50cad62e833) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:49.671535 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Volume volume-c76ca520-9eba-450b-86ae-0323cf62d27e (c76ca520-9eba-450b-86ae-0323cf62d27e): created successfully May 28 21:32:49.675172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (dee969e3-3285-4377-8912-d50cad62e833) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:49.680535 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (3df735d5-b0a3-4d3f-82ac-cf0c4f769637) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:49.786078 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e651508c-5bb4-428e-b92c-6d93fcc45c6a tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:32:49.812007 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] Lock "b'cinder-be8b4c83-8b39-4532-a3ec-dc28247b1e69-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:32:50.005370 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:51.591590 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:32:51.592739 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] Volume volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:32:51.939519 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] deleting rbd volume volume-be8b4c83-8b39-4532-a3ec-dc28247b1e69 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:32:52.096026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:32:52.115665 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (78d282ee-c181-43be-8e84-a25dc1e540b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:52.121874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7264762d-fea3-4a1a-9c65-b6c8df0ca179) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:52.139423 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (7264762d-fea3-4a1a-9c65-b6c8df0ca179) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:32:50Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1804808246',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=97401bf7-af10-4a08-9b10-07c2b6b77681,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:32:51Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:32:51Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:52.144606 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d1804df9-6c9d-422c-8ddb-0547ffd5a9f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:52.150206 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d1804df9-6c9d-422c-8ddb-0547ffd5a9f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:52.153492 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (62e9be54-dbc4-4732-abf1-dc6476a22d15) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:52.157395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (62e9be54-dbc4-4732-abf1-dc6476a22d15) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '97401bf7-af10-4a08-9b10-07c2b6b77681', 'volume_name': 'volume-97401bf7-af10-4a08-9b10-07c2b6b77681', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:52.162721 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a5b705c-62e3-438c-b5ec-0e3a43f65df8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:52.166408 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9a5b705c-62e3-438c-b5ec-0e3a43f65df8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:52.171242 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (39b166ff-3b81-42b9-aea6-e528f4640a9d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:52.172312 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Volume 97401bf7-af10-4a08-9b10-07c2b6b77681: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-97401bf7-af10-4a08-9b10-07c2b6b77681', 'volume_size': 1} May 28 21:32:52.172701 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] creating volume 'volume-97401bf7-af10-4a08-9b10-07c2b6b77681' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:32:52.173363 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:56.426857 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:32:56.427497 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:32:56.430826 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (39b166ff-3b81-42b9-aea6-e528f4640a9d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-97401bf7-af10-4a08-9b10-07c2b6b77681', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:56.435284 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (35619e42-2263-406c-86c9-370628d939c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:32:56.517342 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Volume volume-97401bf7-af10-4a08-9b10-07c2b6b77681 (97401bf7-af10-4a08-9b10-07c2b6b77681): created successfully May 28 21:32:56.521333 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (35619e42-2263-406c-86c9-370628d939c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:32:56.526841 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (78d282ee-c181-43be-8e84-a25dc1e540b7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:32:56.608556 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-26831b02-a224-4af8-92ad-218e3a57d91c tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:32:57.397606 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] Lock "b'cinder-97401bf7-af10-4a08-9b10-07c2b6b77681-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:32:57.487727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:32:57.845543 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:32:57.846470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] Volume volume-97401bf7-af10-4a08-9b10-07c2b6b77681 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:32:58.305236 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-97401bf7-af10-4a08-9b10-07c2b6b77681 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:33:00.958141 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:33:00.958810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:00.979763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:01.002472 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:06.184777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] Created reservations ['89ecbf39-d688-4db3-be66-a59e6d44242a', '33ef345a-e9ff-480a-ade0-5ce96242f718', '7dd9b1c4-5796-415d-8e0c-21eb52980f62', 'b43250bb-e1fe-4a24-8493-33a873ff1133'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:33:06.392222 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:06.414302 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:06.436944 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:06.441284 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] Deleted volume successfully. May 28 21:33:06.537601 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-3ae95691-f8a0-4823-bfc3-4569a1d91179 req-61a0ec40-a88b-4115-bbce-81e2b5ec3982 tempest-VolumesGetTestJSON-675753615 None] Lock "b'cinder-719cf254-e14a-4776-876d-e0177461f78b-delete_volume'" released by "delete_volume" :: held 22.421s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:33:12.551092 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=62,cinder:UPDATE=31,cinder:INSERT=5,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:17.180672 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=33,cinder:UPDATE=14,cinder:DELETE=3,cinder:INSERT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:17.283530 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] Created reservations ['1bdde16e-6bec-459f-9224-9086af544c63', 'e15ef697-de33-4ec9-ae38-993ce14cb8c3', '2a3eafc7-4096-4abc-91a7-e51e96df8561', '82cbec03-0148-429f-a0de-4f7efaa1298e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:33:17.562046 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:17.734514 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:17.769736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:17.774925 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] Deleted volume successfully. May 28 21:33:17.888564 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a9255d7d-a22a-4384-b24d-317ce7e1cb43 tempest-AttachVolumeTestJSON-1447462214 None] Lock "b'cinder-be8b4c83-8b39-4532-a3ec-dc28247b1e69-delete_volume'" released by "delete_volume" :: held 28.076s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:33:18.510386 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] Lock "b'cinder-60d078cc-0a8b-44c0-924e-2d287ff7f3bd-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:33:18.606009 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:21.614959 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:33:21.617550 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] Volume volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:33:22.120594 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] deleting rbd volume volume-60d078cc-0a8b-44c0-924e-2d287ff7f3bd {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:33:24.918716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] Created reservations ['e9dc74df-7e4c-4380-a058-463d3fd2aa90', 'c7c42180-9d8f-4ea9-a253-b69696d2390d', '73a62c08-174d-49df-be7a-49975bc13876', '9a52ea6a-2188-4031-88e8-8e40c073726e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:33:25.175258 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:25.201162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:25.226915 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:25.231625 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:33:25.315869 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d8e04c8b-da96-4134-ac85-dac3503a6492 tempest-VolumesGetTest-658875503 None] Lock "b'cinder-97401bf7-af10-4a08-9b10-07c2b6b77681-delete_volume'" released by "delete_volume" :: held 27.919s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:33:26.109854 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] Lock "b'cinder-c76ca520-9eba-450b-86ae-0323cf62d27e-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:33:26.210647 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:27.175402 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:33:27.176820 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] Volume volume-c76ca520-9eba-450b-86ae-0323cf62d27e is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:33:27.731743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-c76ca520-9eba-450b-86ae-0323cf62d27e {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:33:45.255365 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=39,cinder:INSERT=6,cinder:UPDATE=24,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:45.346332 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] Created reservations ['2ec9432b-8b30-40d0-9d6d-121768da2636', '6d01ff75-2742-4a00-99f2-ffe818c477c2', '995601cd-7051-4fad-878b-0c0770aae31f', '9a7d11c1-76b1-443a-bdc6-d2256625d2d8'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:33:45.551726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:45.577463 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:45.615751 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:45.620101 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] Deleted volume successfully. May 28 21:33:45.726901 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-dc75fcc4-b973-49ad-957a-5195584a5f81 tempest-AttachVolumeTestJSON-1447462214 None] Lock "b'cinder-60d078cc-0a8b-44c0-924e-2d287ff7f3bd-delete_volume'" released by "delete_volume" :: held 27.216s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:33:46.212686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] Lock "b'cinder-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:33:46.359246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:46.975207 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:33:46.975679 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] Volume volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:33:48.125741 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] deleting rbd volume volume-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:33:52.496104 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=40,cinder:UPDATE=27,cinder:INSERT=6,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:33:52.603842 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] Created reservations ['85e4a208-d006-40d8-aff3-ce401ca86941', 'df3b7a56-fd71-47b8-a5b2-7db63b5e2a97', 'ac9d9118-7fc0-4551-bc2d-58da88e4abed', '3d8a0b60-12fc-4db6-afe1-a6e490b3017b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:33:52.768427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:52.802237 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:52.828913 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:52.834635 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:33:52.899193 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5af80c7c-94f9-4232-8bf0-e5a3584df00c tempest-VolumesGetTest-658875503 None] Lock "b'cinder-c76ca520-9eba-450b-86ae-0323cf62d27e-delete_volume'" released by "delete_volume" :: held 26.789s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:33:53.277609 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Terminate volume connection completed successfully. May 28 21:33:53.416494 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Terminate volume connection completed successfully. May 28 21:33:53.432056 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a04865ad-c8ba-46f6-9f7e-2f4250d4cbc5 tempest-TestVolumeBackupRestore-1923413318 None] Remove volume export completed successfully. May 28 21:33:53.442036 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Remove volume export completed successfully. May 28 21:33:53.496775 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-efe7548f-0d36-4ab9-a529-aadaf622854f-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:33:53.510580 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:53.710312 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:33:53.832610 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-efe7548f-0d36-4ab9-a529-aadaf622854f {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:33:54.393427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:33:54.413547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (a456262d-4cde-4966-8e75-dda9976fff6e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:33:54.420899 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a4a65465-3c6a-41a4-a64d-fbeba99f01b1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:54.436760 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a4a65465-3c6a-41a4-a64d-fbeba99f01b1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:33:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-1434304056',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=fed5b816-fae3-4653-abd6-f6a9ce542319,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:33:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:33:54Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:54.442093 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (39cbeca2-2b37-48ef-8b27-280fad91d8bb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:54.446162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (39cbeca2-2b37-48ef-8b27-280fad91d8bb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:54.451844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e1cc3e3d-4240-4051-9d2e-64bf1ff976ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:54.455985 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e1cc3e3d-4240-4051-9d2e-64bf1ff976ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'fed5b816-fae3-4653-abd6-f6a9ce542319', 'volume_name': 'volume-fed5b816-fae3-4653-abd6-f6a9ce542319', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:54.461249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (626f204c-ad04-4e02-a5ba-4f3fc8a7f22a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:54.466112 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (626f204c-ad04-4e02-a5ba-4f3fc8a7f22a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:54.470929 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ddeb5c8d-6e4d-47a5-9abf-0e0c36ad1332) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:54.472207 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Volume fed5b816-fae3-4653-abd6-f6a9ce542319: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-fed5b816-fae3-4653-abd6-f6a9ce542319', 'volume_size': 1} May 28 21:33:54.472403 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] creating volume 'volume-fed5b816-fae3-4653-abd6-f6a9ce542319' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:33:54.473153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:55.204999 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:33:55.206369 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:33:55.210743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ddeb5c8d-6e4d-47a5-9abf-0e0c36ad1332) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-fed5b816-fae3-4653-abd6-f6a9ce542319', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:55.216887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (116e73a0-cfb0-46b5-9fa0-ccf85884a79d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:55.300372 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Volume volume-fed5b816-fae3-4653-abd6-f6a9ce542319 (fed5b816-fae3-4653-abd6-f6a9ce542319): created successfully May 28 21:33:55.304077 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (116e73a0-cfb0-46b5-9fa0-ccf85884a79d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:55.316895 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (a456262d-4cde-4966-8e75-dda9976fff6e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:33:55.391968 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-4b81bfe0-2cf4-40cf-80a8-4e83d6faa652 tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:33:55.597789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] Created reservations ['f6320494-d9b6-4c14-bdb4-d3d9797fa5f5', 'ac27829e-84d2-4654-b366-8622daba4959', '2a0ba1a2-801d-4e5a-a26d-c91b4ab47301', 'e4d6295c-493a-47e4-8cc4-2fa381b9ddb4'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:33:55.769274 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:55.819498 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:55.858637 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:33:55.865672 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] Deleted volume successfully. May 28 21:33:55.973280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-b706b79d-bbd9-45a6-adbf-eae4253d40ce tempest-AttachVolumeTestJSON-1447462214 None] Lock "b'cinder-ed95b6dd-f6d9-4cde-a354-d6ae7327ae42-delete_volume'" released by "delete_volume" :: held 9.761s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:33:57.382452 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:33:57.428512 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (c47e766f-4676-4551-b2d2-c5adb7a5b600) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:33:57.435258 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (15e67599-d51e-4e5e-9427-160b26e64f38) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:57.451897 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (15e67599-d51e-4e5e-9427-160b26e64f38) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:33:56Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-560266929',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=20d67167-4aa6-41c5-8af8-1c6d97a2d484,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:33:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=fed5b816-fae3-4653-abd6-f6a9ce542319,status='creating',terminated_at=None,updated_at=2022-05-28T21:33:57Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:57.457644 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (430048b4-10c9-45a7-a180-8df600e4963a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:57.462109 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (430048b4-10c9-45a7-a180-8df600e4963a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:57.468783 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b2a3a045-5d63-4b17-a076-8f5b68fac626) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:57.488692 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b2a3a045-5d63-4b17-a076-8f5b68fac626) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '20d67167-4aa6-41c5-8af8-1c6d97a2d484', 'volume_name': 'volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484', 'volume_size': 1, 'source_volid': 'fed5b816-fae3-4653-abd6-f6a9ce542319', 'source_volstatus': 'available'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:57.495733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (07854d62-5faf-4b7b-823a-6d83141a5d82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:57.501207 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (07854d62-5faf-4b7b-823a-6d83141a5d82) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:33:57.506417 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c5c17df8-f375-4ffa-aa65-5852a81513c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:33:57.508073 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Volume 20d67167-4aa6-41c5-8af8-1c6d97a2d484: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484', 'volume_size': 1, 'source_volid': 'fed5b816-fae3-4653-abd6-f6a9ce542319', 'source_volstatus': 'available'} May 28 21:33:57.522960 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:33:57.801942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] creating snapshot='volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:34:01.074228 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] cloning 'volume-fed5b816-fae3-4653-abd6-f6a9ce542319@volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484.clone_snap' to 'volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:34:01.075830 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:34:01.076398 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:01.128496 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:01.166087 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:34:01.294613 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:34:01.299050 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1497bf42-a2f5-4e0c-ad17-e1919a913240 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-efe7548f-0d36-4ab9-a529-aadaf622854f-delete_volume'" released by "delete_volume" :: held 7.802s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:03.382530 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Volume volume-fed5b816-fae3-4653-abd6-f6a9ce542319 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:03.383713 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:34:03.384085 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:34:04.140239 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:34:04.143976 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c5c17df8-f375-4ffa-aa65-5852a81513c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484', 'volume_size': 1, 'source_volid': 'fed5b816-fae3-4653-abd6-f6a9ce542319', 'source_volstatus': 'available'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:04.149115 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e00d95ae-0b18-488d-bffa-4742ccdae360) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:04.201857 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Volume volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484 (20d67167-4aa6-41c5-8af8-1c6d97a2d484): created successfully May 28 21:34:04.205450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e00d95ae-0b18-488d-bffa-4742ccdae360) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:04.215918 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (c47e766f-4676-4551-b2d2-c5adb7a5b600) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:04.289505 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8176c0dc-14ca-42b9-9654-98bc52759ab7 tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:34:04.468946 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b2d650b5-8c84-4abe-9a17-b3d4a071b0d0 req-af1e090c-fc3b-4667-9814-e56bebe3e1b4 tempest-TestVolumeBackupRestore-1923413318 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:34:05.469130 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-b2d650b5-8c84-4abe-9a17-b3d4a071b0d0 req-af1e090c-fc3b-4667-9814-e56bebe3e1b4 tempest-TestVolumeBackupRestore-1923413318 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.995s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:34:05.469130 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-b2d650b5-8c84-4abe-9a17-b3d4a071b0d0 req-af1e090c-fc3b-4667-9814-e56bebe3e1b4 tempest-TestVolumeBackupRestore-1923413318 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:34:05.614973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-b2d650b5-8c84-4abe-9a17-b3d4a071b0d0 req-af1e090c-fc3b-4667-9814-e56bebe3e1b4 tempest-TestVolumeBackupRestore-1923413318 None] Connection info returned from driver {'name': 'volumes/volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'be2c4092-4534-4aee-adba-6c13e90832cc'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:34:05.841998 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=59,cinder:INSERT=6,cinder:UPDATE=33,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:05.841998 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-b2d650b5-8c84-4abe-9a17-b3d4a071b0d0 req-af1e090c-fc3b-4667-9814-e56bebe3e1b4 tempest-TestVolumeBackupRestore-1923413318 None] attachment_update completed successfully. May 28 21:34:06.186420 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:34:06.233331 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (3a1c7c78-a410-4890-9e65-9aad25c9c537) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:06.241580 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6f64fb19-e0fa-4f56-8eca-bed67d37697c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:06.282444 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (6f64fb19-e0fa-4f56-8eca-bed67d37697c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:34:05Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1643224128',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=79bd4597-347f-4629-9b7f-754f992d0263,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:34:06Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:34:06Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:06.286941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c7ff21e3-4b50-44f7-9f7b-fa47a5694be9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:06.292281 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c7ff21e3-4b50-44f7-9f7b-fa47a5694be9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:06.298495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fa36c865-bccb-438b-bc56-e740a3f21447) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:06.305464 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (fa36c865-bccb-438b-bc56-e740a3f21447) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '79bd4597-347f-4629-9b7f-754f992d0263', 'volume_name': 'volume-79bd4597-347f-4629-9b7f-754f992d0263', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:06.312317 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (09a796eb-07bc-415c-8e37-97277ea0e6f0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:06.317086 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (09a796eb-07bc-415c-8e37-97277ea0e6f0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:06.328800 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dbb9182b-3adf-4217-96bb-0526431af7c0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:06.328800 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Volume 79bd4597-347f-4629-9b7f-754f992d0263: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-79bd4597-347f-4629-9b7f-754f992d0263', 'volume_size': 1} May 28 21:34:06.328800 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] creating volume 'volume-79bd4597-347f-4629-9b7f-754f992d0263' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:34:06.328800 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:07.470502 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:34:07.471986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:34:07.476135 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dbb9182b-3adf-4217-96bb-0526431af7c0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-79bd4597-347f-4629-9b7f-754f992d0263', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:07.483297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (61871e96-919f-45be-98a4-79aa4c5228b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:07.529680 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Volume volume-79bd4597-347f-4629-9b7f-754f992d0263 (79bd4597-347f-4629-9b7f-754f992d0263): created successfully May 28 21:34:07.537518 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (61871e96-919f-45be-98a4-79aa4c5228b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:07.548617 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (3a1c7c78-a410-4890-9e65-9aad25c9c537) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:07.631700 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1b1c08a6-a1a3-45b0-a903-0a313f765093 tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:34:08.425591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] Lock "b'cinder-79bd4597-347f-4629-9b7f-754f992d0263-delete_volume'" acquired by "delete_volume" :: waited 0.024s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:08.546876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:08.877454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:34:08.877926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] Volume volume-79bd4597-347f-4629-9b7f-754f992d0263 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:09.131845 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-79bd4597-347f-4629-9b7f-754f992d0263 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:34:12.837681 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] Created reservations ['dd0cd41d-1b4d-4714-a7f4-7786f39f2f75', '6562a2e6-10ee-496f-9ddc-ecfa2c4fc0ea', 'f280c929-ba61-4478-a29c-35245a04de1e', '09b11648-85db-4230-b2b8-484f936f3b7b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:34:13.067466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:13.107868 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:13.165375 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:34:13.177547 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:34:13.253874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8c875381-2a39-49ce-b346-0f03ea3e3c7d tempest-VolumesGetTest-658875503 None] Lock "b'cinder-79bd4597-347f-4629-9b7f-754f992d0263-delete_volume'" released by "delete_volume" :: held 4.828s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:14.023004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] Lock "b'cinder-20d67167-4aa6-41c5-8af8-1c6d97a2d484-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:14.110061 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=63,cinder:INSERT=4,cinder:UPDATE=29,cinder:DELETE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:14.127739 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:14.317274 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:34:14.422886 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:34:18.591768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:34:18.770250 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] Volume volume-fed5b816-fae3-4653-abd6-f6a9ce542319 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:18.771242 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] deleting parent snapshot volume-20d67167-4aa6-41c5-8af8-1c6d97a2d484.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:34:21.665912 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] Created reservations ['a3c13454-bcac-4381-8a83-4bbaa346a788', '20adb84e-bdf7-4773-9aa1-901437c963eb', '2a2c0d21-317c-465c-8388-ebf599cd80b0', 'bc965906-9b04-4574-a85e-0d81789ba535'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:34:21.946408 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:21.988420 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:22.037500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:34:22.043398 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:34:22.145812 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-22825c01-a738-47fb-bc29-21204aed151c tempest-VolumesGetTest-658875503 None] Lock "b'cinder-20d67167-4aa6-41c5-8af8-1c6d97a2d484-delete_volume'" released by "delete_volume" :: held 8.123s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:23.484762 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:34:23.504694 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (779cfd02-2e56-4626-ac88-d76b8cf93a80) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:23.511393 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (635135df-264d-4463-8fbb-a39ae91a6bb0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:23.528958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (635135df-264d-4463-8fbb-a39ae91a6bb0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:34:23Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTest-Volume-736192115',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=20d36d19-83e8-4ce8-ba22-c069986113da,launched_at=None,metadata={Type='Test'},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:34:23Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:34:23Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:23.533819 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (31e8e0b3-8d6e-4326-8549-06a9b680df28) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:23.537931 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (31e8e0b3-8d6e-4326-8549-06a9b680df28) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:23.543574 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (abadb25f-b75f-4808-bbb4-ef3dd66d8a82) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:23.654985 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (abadb25f-b75f-4808-bbb4-ef3dd66d8a82) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '20d36d19-83e8-4ce8-ba22-c069986113da', 'volume_name': 'volume-20d36d19-83e8-4ce8-ba22-c069986113da', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:23.661172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b918e6d7-7088-45cb-8d7f-cefc988ef4d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:23.665186 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b918e6d7-7088-45cb-8d7f-cefc988ef4d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:23.676905 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ecb23ec1-8b75-4f7e-a815-d9ed64d04479) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:23.676905 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Volume 20d36d19-83e8-4ce8-ba22-c069986113da: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-20d36d19-83e8-4ce8-ba22-c069986113da', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:34:23.678906 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Cloning 20d36d19-83e8-4ce8-ba22-c069986113da from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:34:23.689217 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.016s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:23.756275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 34, 23, 693413)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:34:23.756738 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:34:23.757167 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:34:23.762600 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.073s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:23.765902 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:34:23.769935 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 34, 23, 767598)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:34:23.770352 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:34:23.770743 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:34:23.782212 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:23.997429 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] creating snapshot='volume-20d36d19-83e8-4ce8-ba22-c069986113da.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:34:25.304140 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-20d36d19-83e8-4ce8-ba22-c069986113da.clone_snap' to 'volume-20d36d19-83e8-4ce8-ba22-c069986113da' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:34:25.432881 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:34:25.452145 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (7fe120b7-0883-42f7-a7be-00450a9c33f9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:25.458449 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (37744046-67a8-459c-92e3-ec7da096a637) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:25.474434 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (37744046-67a8-459c-92e3-ec7da096a637) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:34:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1997750264',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a420983a-b71a-4ca0-935a-b83c5153223d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6ed82f578e214948a9311cbdbb55a027',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:34:24Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:34:24Z,use_quota=True,user_id='8d033e99f9294e348b01c59a970931d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:25.479686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff37a316-f571-4111-bfa8-207044d9bd9b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:25.483771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ff37a316-f571-4111-bfa8-207044d9bd9b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:25.488723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (35b8e77a-aa4f-489e-8326-45a2f905b7ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:25.577658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (35b8e77a-aa4f-489e-8326-45a2f905b7ab) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'a420983a-b71a-4ca0-935a-b83c5153223d', 'volume_name': 'volume-a420983a-b71a-4ca0-935a-b83c5153223d', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:25.582782 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a62a6daa-070e-42cc-86d1-6764609ca0e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:25.586753 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a62a6daa-070e-42cc-86d1-6764609ca0e9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:25.593628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84579df7-dbfd-4d97-8cf5-140540fb9dc1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:25.594977 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Volume a420983a-b71a-4ca0-935a-b83c5153223d: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-a420983a-b71a-4ca0-935a-b83c5153223d', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:34:25.599962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Cloning a420983a-b71a-4ca0-935a-b83c5153223d from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:34:25.603591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.007s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:25.693427 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 34, 25, 608460)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:34:25.693773 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:34:25.696234 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:34:25.699019 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.095s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:25.700283 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:34:25.706936 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 34, 25, 704141)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:34:25.707348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:34:25.707792 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:34:25.720348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:26.048328 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] creating snapshot='volume-a420983a-b71a-4ca0-935a-b83c5153223d.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:34:28.087866 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-a420983a-b71a-4ca0-935a-b83c5153223d.clone_snap' to 'volume-a420983a-b71a-4ca0-935a-b83c5153223d' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:34:28.134580 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:28.136229 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:34:28.136604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:34:28.959220 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:34:28.961034 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Marking volume 20d36d19-83e8-4ce8-ba22-c069986113da as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:34:29.020204 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 20d36d19-83e8-4ce8-ba22-c069986113da. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:34:29.020632 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Creating volume glance metadata for volume 20d36d19-83e8-4ce8-ba22-c069986113da backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:34:29.179603 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ecb23ec1-8b75-4f7e-a815-d9ed64d04479) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-20d36d19-83e8-4ce8-ba22-c069986113da', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:29.187149 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7fa4fba1-a39b-433c-b621-ae6ed0789c01) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:29.253805 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Volume volume-20d36d19-83e8-4ce8-ba22-c069986113da (20d36d19-83e8-4ce8-ba22-c069986113da): created successfully May 28 21:34:29.258004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (7fa4fba1-a39b-433c-b621-ae6ed0789c01) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:29.264181 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (779cfd02-2e56-4626-ac88-d76b8cf93a80) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:29.333168 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9e37908a-1cac-4788-970c-5d73f7d6637a tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:34:29.526871 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:29.529308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:34:29.529688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:34:29.682302 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:34:29.683016 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Marking volume a420983a-b71a-4ca0-935a-b83c5153223d as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:34:29.878491 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to a420983a-b71a-4ca0-935a-b83c5153223d. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:34:29.878919 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Creating volume glance metadata for volume a420983a-b71a-4ca0-935a-b83c5153223d backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:34:30.061696 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (84579df7-dbfd-4d97-8cf5-140540fb9dc1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a420983a-b71a-4ca0-935a-b83c5153223d', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:30.067145 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (806e4d1f-e31f-44b7-96bc-f5062dac0715) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:30.174465 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-a420983a-b71a-4ca0-935a-b83c5153223d (a420983a-b71a-4ca0-935a-b83c5153223d): created successfully May 28 21:34:30.181492 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (806e4d1f-e31f-44b7-96bc-f5062dac0715) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:30.190631 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (7fe120b7-0883-42f7-a7be-00450a9c33f9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:30.266460 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-7f615b68-528b-44a2-bfa9-5463006bcf46 tempest-VolumesSnapshotTestJSON-961770574 None] Created volume successfully. May 28 21:34:30.931078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:34:30.951776 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (432a56c1-eb74-41d3-8c0c-89ccaf1c3a6e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:30.958354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c8712820-0326-4db3-9ad8-59f1ce917f40) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:30.974552 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c8712820-0326-4db3-9ad8-59f1ce917f40) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:34:30Z,deleted=False,deleted_at=None,display_description='tempest-@#$%^* description-1622192045',display_name=None,ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=71e7bd22-7c5b-439f-a57e-59b8a2b60471,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='045cd28b35b3467e91274fff8140771b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:34:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:34:31Z,use_quota=True,user_id='1d9d97eaf9f74d58adf5ea7e90918c22',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:30.979847 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (104d5b80-6642-44e9-acff-2322755fd438) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:30.983966 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (104d5b80-6642-44e9-acff-2322755fd438) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:30.989034 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ed6ff135-61b0-4c4e-89f4-db306933f637) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:30.993184 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ed6ff135-61b0-4c4e-89f4-db306933f637) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '71e7bd22-7c5b-439f-a57e-59b8a2b60471', 'volume_name': 'volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:30.998612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (678f00a3-a6cf-44af-982d-b171d8401ac4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:31.002494 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (678f00a3-a6cf-44af-982d-b171d8401ac4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:31.007130 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b77234e-c9ce-4ae7-a364-736d43a77256) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:31.008267 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Volume 71e7bd22-7c5b-439f-a57e-59b8a2b60471: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471', 'volume_size': 1} May 28 21:34:31.008659 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] creating volume 'volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:34:31.009419 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:31.672753 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=65,cinder:INSERT=13,cinder:UPDATE=21,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:31.675549 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-095fe1e7-4db6-46a0-b58b-2fef32469503 req-7c173ed5-e19d-4d31-b75a-bec0aeecd991 tempest-VolumesSnapshotTestJSON-961770574 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:34:31.841693 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:34:31.842098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:34:31.846779 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3b77234e-c9ce-4ae7-a364-736d43a77256) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:31.852557 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89cd4ee5-8d74-4663-89ff-e8998eeadd2a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:34:31.974334 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Volume volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471 (71e7bd22-7c5b-439f-a57e-59b8a2b60471): created successfully May 28 21:34:31.977946 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89cd4ee5-8d74-4663-89ff-e8998eeadd2a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:34:31.984018 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Flow 'volume_create_manager' (432a56c1-eb74-41d3-8c0c-89ccaf1c3a6e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:34:32.133520 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-524d2874-314e-494e-8040-a316d516c4d7 tempest-VolumesGetTest-658875503 None] Created volume successfully. May 28 21:34:32.406681 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-095fe1e7-4db6-46a0-b58b-2fef32469503 req-7c173ed5-e19d-4d31-b75a-bec0aeecd991 tempest-VolumesSnapshotTestJSON-961770574 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.731s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:34:32.407371 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-095fe1e7-4db6-46a0-b58b-2fef32469503 req-7c173ed5-e19d-4d31-b75a-bec0aeecd991 tempest-VolumesSnapshotTestJSON-961770574 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a420983a-b71a-4ca0-935a-b83c5153223d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'a420983a-b71a-4ca0-935a-b83c5153223d', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:34:32.541950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-095fe1e7-4db6-46a0-b58b-2fef32469503 req-7c173ed5-e19d-4d31-b75a-bec0aeecd991 tempest-VolumesSnapshotTestJSON-961770574 None] Connection info returned from driver {'name': 'volumes/volume-a420983a-b71a-4ca0-935a-b83c5153223d', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a420983a-b71a-4ca0-935a-b83c5153223d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '9d7d5980-cfd9-44d3-a11a-a2087fcde8dd'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:34:32.684799 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-095fe1e7-4db6-46a0-b58b-2fef32469503 req-7c173ed5-e19d-4d31-b75a-bec0aeecd991 tempest-VolumesSnapshotTestJSON-961770574 None] attachment_update completed successfully. May 28 21:34:33.145325 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] Lock "b'cinder-71e7bd22-7c5b-439f-a57e-59b8a2b60471-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:33.250825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:33.375014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:34:33.375514 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] Volume volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:33.523358 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-71e7bd22-7c5b-439f-a57e-59b8a2b60471 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:34:36.194133 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=59,cinder:INSERT=17,cinder:UPDATE=21,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:36.339093 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-048e9186-6224-4d05-a56d-f6b1773e2292 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:38.511523 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] Created reservations ['9160e3aa-7295-4638-844f-0ce7ec7298c5', 'cb4976f7-46e7-4ff9-b011-fad211472ed7', '3f752096-fc04-40b7-8536-fe5d23328119', '6c4f2a06-bd29-42ea-9de1-d66ac7aa052b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:34:38.775127 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:38.797486 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:38.821208 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:34:38.825428 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:34:38.909192 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1d917c78-5b6d-4f8a-9f15-27e68884a9ac tempest-VolumesGetTest-658875503 None] Lock "b'cinder-71e7bd22-7c5b-439f-a57e-59b8a2b60471-delete_volume'" released by "delete_volume" :: held 5.764s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:39.070367 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-048e9186-6224-4d05-a56d-f6b1773e2292 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:34:39.658364 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] Lock "b'cinder-20d36d19-83e8-4ce8-ba22-c069986113da-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:39.782341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f1ebc548-0004-4451-88ca-22b0d64a8edf tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:40.036218 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:40.438352 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:34:40.568593 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-20d36d19-83e8-4ce8-ba22-c069986113da {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:34:43.460331 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f1ebc548-0004-4451-88ca-22b0d64a8edf tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:34:44.371123 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f64dc8eb-f58d-4f33-9495-fe48705fefdf tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:48.096032 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:34:48.369849 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] deleting parent snapshot volume-20d36d19-83e8-4ce8-ba22-c069986113da.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:34:53.031489 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=67,cinder:UPDATE=14,cinder:INSERT=18,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:53.123008 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] Created reservations ['fd30c51d-fee1-4938-be7d-727733c706d4', '0f3dd803-8b32-46b4-becf-f0058c0d7793', '88def646-ea63-4789-a84e-59a5e70c7469', '1370b699-7a8e-41f1-8d52-467fa11172bb'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:34:53.214636 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f64dc8eb-f58d-4f33-9495-fe48705fefdf tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:34:53.413683 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:53.464585 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:53.493953 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:34:53.498359 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:34:53.502850 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=44,cinder:INSERT=26,cinder:UPDATE=26,cinder:DELETE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:34:53.576777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e5dc83de-4314-4bbb-a007-4d8090044835 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-fa3b8ab8-2f21-4428-b045-3bbfab58e4b3-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:53.578501 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e5dc83de-4314-4bbb-a007-4d8090044835 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:53.913572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8df3be5a-0cad-46c9-8abd-a28b3d83396c tempest-VolumesGetTest-658875503 None] Lock "b'cinder-20d36d19-83e8-4ce8-ba22-c069986113da-delete_volume'" released by "delete_volume" :: held 14.256s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:34:53.919552 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] Lock "b'cinder-fed5b816-fae3-4653-abd6-f6a9ce542319-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:34:54.031400 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:34:54.306109 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:34:54.306771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] Volume volume-fed5b816-fae3-4653-abd6-f6a9ce542319 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:34:54.534588 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] deleting rbd volume volume-fed5b816-fae3-4653-abd6-f6a9ce542319 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:34:59.957360 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] Created reservations ['aaecc9b6-e147-437f-86e9-96aecb9b7221', 'b4914d13-7722-46fa-9830-419e71544bd5', '8d4792f6-9578-49b8-b837-9560e56d7da6', '5628da9a-e1c3-4dc7-be72-48c1f65349a4'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:35:00.157827 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:00.187812 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:00.221628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:35:00.231244 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] Deleted volume successfully. May 28 21:35:00.281714 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-937d8f91-a613-4b77-b076-b7dd2e45caf9 tempest-VolumesGetTest-658875503 None] Lock "b'cinder-fed5b816-fae3-4653-abd6-f6a9ce542319-delete_volume'" released by "delete_volume" :: held 6.362s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:35:01.079025 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:35:01.081175 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:01.162733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:01.186885 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:35:05.478254 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:35:05.501329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Flow 'volume_create_manager' (589cf4e3-289b-4ec6-ae4a-a77f78cc542a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:35:05.509873 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (47b5b87c-ecff-4263-9fd4-eceb55a3a620) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:05.529006 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (47b5b87c-ecff-4263-9fd4-eceb55a3a620) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:35:05Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-1229110366',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=9ae207c8-8c87-4201-942c-781ab5b960e5,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c9caba58394e1480acb632405111c5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:35:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:35:05Z,use_quota=True,user_id='b16d0e791edd4f119e646e7ced74bf74',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:05.536339 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (20fc14c4-b7a3-4b27-abb5-40b9de5081ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:05.542359 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (20fc14c4-b7a3-4b27-abb5-40b9de5081ab) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:05.549745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5abc3159-0e76-46c0-ae3d-992087526929) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:05.555141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5abc3159-0e76-46c0-ae3d-992087526929) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9ae207c8-8c87-4201-942c-781ab5b960e5', 'volume_name': 'volume-9ae207c8-8c87-4201-942c-781ab5b960e5', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:05.560166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ee0b81a3-ba35-4463-ade7-de57c115c8e3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:05.564590 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ee0b81a3-ba35-4463-ade7-de57c115c8e3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:05.570486 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3fa69ab4-3aa6-44e7-8dc6-6cf735d533d8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:05.571655 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Volume 9ae207c8-8c87-4201-942c-781ab5b960e5: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9ae207c8-8c87-4201-942c-781ab5b960e5', 'volume_size': 1} May 28 21:35:05.572044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] creating volume 'volume-9ae207c8-8c87-4201-942c-781ab5b960e5' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:35:05.572820 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:06.601342 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:35:06.602832 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:35:06.607277 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3fa69ab4-3aa6-44e7-8dc6-6cf735d533d8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9ae207c8-8c87-4201-942c-781ab5b960e5', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:06.622072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6d360447-7803-40e4-83c0-55b04cf52bd3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:06.693120 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Volume volume-9ae207c8-8c87-4201-942c-781ab5b960e5 (9ae207c8-8c87-4201-942c-781ab5b960e5): created successfully May 28 21:35:06.699314 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (6d360447-7803-40e4-83c0-55b04cf52bd3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:06.705050 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Flow 'volume_create_manager' (589cf4e3-289b-4ec6-ae4a-a77f78cc542a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:35:06.820277 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d7ac7990-e969-4e77-8a2e-061e823717a0 tempest-AttachVolumeNegativeTest-679324601 None] Created volume successfully. May 28 21:35:08.368124 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d77fc369-1c41-48f8-b600-a0882af34fe1 req-8280f68c-84cf-4465-aabf-c2580cd2a8f2 tempest-AttachVolumeNegativeTest-679324601 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:35:09.245188 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d77fc369-1c41-48f8-b600-a0882af34fe1 req-8280f68c-84cf-4465-aabf-c2580cd2a8f2 tempest-AttachVolumeNegativeTest-679324601 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.878s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:35:09.246274 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-d77fc369-1c41-48f8-b600-a0882af34fe1 req-8280f68c-84cf-4465-aabf-c2580cd2a8f2 tempest-AttachVolumeNegativeTest-679324601 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-9ae207c8-8c87-4201-942c-781ab5b960e5', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '9ae207c8-8c87-4201-942c-781ab5b960e5', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:35:09.379193 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-d77fc369-1c41-48f8-b600-a0882af34fe1 req-8280f68c-84cf-4465-aabf-c2580cd2a8f2 tempest-AttachVolumeNegativeTest-679324601 None] Connection info returned from driver {'name': 'volumes/volume-9ae207c8-8c87-4201-942c-781ab5b960e5', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '9ae207c8-8c87-4201-942c-781ab5b960e5', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'f726fa80-2a19-4ce9-a14e-d240d214734c'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:35:09.493504 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-d77fc369-1c41-48f8-b600-a0882af34fe1 req-8280f68c-84cf-4465-aabf-c2580cd2a8f2 tempest-AttachVolumeNegativeTest-679324601 None] attachment_update completed successfully. May 28 21:35:22.991864 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=33,cinder:UPDATE=13,cinder:INSERT=2,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:25.208410 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=31,cinder:INSERT=2,cinder:UPDATE=12,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:25.323150 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e5dc83de-4314-4bbb-a007-4d8090044835 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['3e7f0759-1c46-4e34-99c7-3a59a12c4a8a', '5b92b0aa-b1df-4be8-a04d-89c66add2202', '9eceecb6-2d0a-4f2b-b443-df6397af33b3', '1de8534e-3b5d-4612-a88e-ffdee2ee4f2a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:35:25.631301 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e5dc83de-4314-4bbb-a007-4d8090044835 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:35:25.635984 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e5dc83de-4314-4bbb-a007-4d8090044835 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-fa3b8ab8-2f21-4428-b045-3bbfab58e4b3-delete_snapshot'" released by "delete_snapshot" :: held 32.059s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:35:25.969686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-bccaf67f-5e44-4daf-9760-60ae76eea013 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-28e30c93-0864-4e99-913a-5bbc7a23caa4-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:35:25.971769 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-bccaf67f-5e44-4daf-9760-60ae76eea013 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:42.993776 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:55.010541 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=22,cinder:INSERT=4,cinder:UPDATE=14 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:35:55.083165 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:35:55.104516 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (f3c8db66-2433-4dc3-ab0e-56480bc09761) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:35:55.111172 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (86bceafc-474d-478c-b53d-a6be89801635) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:55.127097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (86bceafc-474d-478c-b53d-a6be89801635) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:35:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-15908415',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ec5a0831-5a82-4255-821f-f73fb72a98be,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='28f8fcc4afe8471c94db58bef7526459',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:35:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:35:55Z,use_quota=True,user_id='b8372df2880a483e8f1b2b1b667ddf04',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:55.133264 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ffce63be-9273-4e87-a836-ed2f5d12431d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:55.136565 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ffce63be-9273-4e87-a836-ed2f5d12431d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:55.142037 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (22445329-39fc-4513-b96a-a55962da41f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:55.146196 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (22445329-39fc-4513-b96a-a55962da41f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ec5a0831-5a82-4255-821f-f73fb72a98be', 'volume_name': 'volume-ec5a0831-5a82-4255-821f-f73fb72a98be', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:55.151719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8d6d37b2-3fa6-48f5-9584-e975d958f5eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:55.155922 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8d6d37b2-3fa6-48f5-9584-e975d958f5eb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:55.160548 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4dac419-c3fd-4664-bcc1-2d8268017b20) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:55.161680 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Volume ec5a0831-5a82-4255-821f-f73fb72a98be: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ec5a0831-5a82-4255-821f-f73fb72a98be', 'volume_size': 1} May 28 21:35:55.162095 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] creating volume 'volume-ec5a0831-5a82-4255-821f-f73fb72a98be' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:35:55.162838 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:55.953437 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:35:55.953820 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:35:55.957591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d4dac419-c3fd-4664-bcc1-2d8268017b20) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ec5a0831-5a82-4255-821f-f73fb72a98be', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:55.962783 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c8e8fd39-7d22-4cd0-aa3f-860d652fd8f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:35:56.081269 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Volume volume-ec5a0831-5a82-4255-821f-f73fb72a98be (ec5a0831-5a82-4255-821f-f73fb72a98be): created successfully May 28 21:35:56.084942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (c8e8fd39-7d22-4cd0-aa3f-860d652fd8f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:35:56.091268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (f3c8db66-2433-4dc3-ab0e-56480bc09761) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:35:56.185505 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e307e2aa-0297-4d9e-9517-f48b304f2e5d tempest-VolumesActionsTest-1083035635 None] Created volume successfully. May 28 21:35:57.029022 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6f709fa7-69cf-4dac-8a0c-68cb6985ef47 tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be'" acquired by "attach_volume" :: waited 0.008s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:35:57.197098 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6f709fa7-69cf-4dac-8a0c-68cb6985ef47 tempest-VolumesActionsTest-1083035635 None] Attaching volume ec5a0831-5a82-4255-821f-f73fb72a98be to instance 374901ee-a9de-43dc-94a5-5752fb696b5e at mountpoint /dev/sdb on host None. May 28 21:35:57.294823 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6f709fa7-69cf-4dac-8a0c-68cb6985ef47 tempest-VolumesActionsTest-1083035635 None] Attach volume completed successfully. May 28 21:35:57.305293 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-6f709fa7-69cf-4dac-8a0c-68cb6985ef47 tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be'" released by "attach_volume" :: held 0.276s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:35:57.414386 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-bccaf67f-5e44-4daf-9760-60ae76eea013 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['da1e6bff-d64e-4d6e-a326-965ba6202d51', '8bc600b1-2643-4803-ad2b-43a886d134be', '87967838-27b8-4f53-8667-fb9ff02e4d19', '6b94bc4c-a954-4e2e-bf12-aaa1151f00d8'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:35:57.671231 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-bccaf67f-5e44-4daf-9760-60ae76eea013 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:35:57.675864 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-bccaf67f-5e44-4daf-9760-60ae76eea013 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-28e30c93-0864-4e99-913a-5bbc7a23caa4-delete_snapshot'" released by "delete_snapshot" :: held 31.707s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:35:57.777415 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2ae60ab8-ecdf-4b23-9675-3370227fbcd3 tempest-VolumesActionsTest-1662161443 None] Terminate volume connection completed successfully. May 28 21:35:57.806662 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2ae60ab8-ecdf-4b23-9675-3370227fbcd3 tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be-detach_volume'" acquired by "detach_volume" :: waited 0.008s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:35:57.818440 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2ae60ab8-ecdf-4b23-9675-3370227fbcd3 tempest-VolumesActionsTest-1662161443 None] Detaching volume ec5a0831-5a82-4255-821f-f73fb72a98be from instance 374901ee-a9de-43dc-94a5-5752fb696b5e. May 28 21:35:58.024173 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2ae60ab8-ecdf-4b23-9675-3370227fbcd3 tempest-VolumesActionsTest-1662161443 None] Detach volume completed successfully. May 28 21:35:58.028498 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2ae60ab8-ecdf-4b23-9675-3370227fbcd3 tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be-detach_volume'" released by "detach_volume" :: held 0.222s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:35:58.077569 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7572a51b-e644-45e8-a3ff-5780374e8390 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-b87f39f6-7332-42a8-baeb-0ad2a8a17b41-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.022s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:35:58.080127 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7572a51b-e644-45e8-a3ff-5780374e8390 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:35:58.275316 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8cc51850-abba-4dc0-8d15-9b95af6615ba tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be-detach_volume'" acquired by "detach_volume" :: waited 0.008s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:35:58.275885 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8cc51850-abba-4dc0-8d15-9b95af6615ba tempest-VolumesActionsTest-1083035635 None] Volume detach called, but volume not attached. May 28 21:35:58.285643 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-8cc51850-abba-4dc0-8d15-9b95af6615ba tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be-detach_volume'" released by "detach_volume" :: held 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:01.086951 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:36:01.086951 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:01.104393 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:01.127246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:02.912609 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:36:02.931185 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (9ee6be0d-d771-403e-997b-2aad27487e29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:02.937923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (35df10b1-09e4-4d5d-a6db-f90174b95296) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:02.953958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (35df10b1-09e4-4d5d-a6db-f90174b95296) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:36:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-367525322',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=185d8769-1e10-42aa-b760-4b50369dfbe7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='28f8fcc4afe8471c94db58bef7526459',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:36:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:36:03Z,use_quota=True,user_id='b8372df2880a483e8f1b2b1b667ddf04',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:02.959387 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (969232da-4739-491f-b432-9a1bc4e4719a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:02.964181 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (969232da-4739-491f-b432-9a1bc4e4719a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:02.969562 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d4fd414-da0e-4b6b-8477-2af8e30942b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:02.973727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4d4fd414-da0e-4b6b-8477-2af8e30942b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '185d8769-1e10-42aa-b760-4b50369dfbe7', 'volume_name': 'volume-185d8769-1e10-42aa-b760-4b50369dfbe7', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:02.978673 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (92fb2f11-049e-4fdd-80b0-37fc564e7189) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:02.982571 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (92fb2f11-049e-4fdd-80b0-37fc564e7189) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:02.987273 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bea96107-8684-41ac-93ed-8f43367136d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:02.988339 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Volume 185d8769-1e10-42aa-b760-4b50369dfbe7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-185d8769-1e10-42aa-b760-4b50369dfbe7', 'volume_size': 1} May 28 21:36:02.988732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] creating volume 'volume-185d8769-1e10-42aa-b760-4b50369dfbe7' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:36:02.989495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:03.765473 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:36:03.765865 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:36:03.769560 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bea96107-8684-41ac-93ed-8f43367136d5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-185d8769-1e10-42aa-b760-4b50369dfbe7', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:03.774392 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (290b6510-2b03-4fdf-be98-010fc3db813c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:03.859873 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Volume volume-185d8769-1e10-42aa-b760-4b50369dfbe7 (185d8769-1e10-42aa-b760-4b50369dfbe7): created successfully May 28 21:36:03.863379 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (290b6510-2b03-4fdf-be98-010fc3db813c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:03.869361 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (9ee6be0d-d771-403e-997b-2aad27487e29) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:03.956942 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-c44b76c6-7fc1-4b8e-87cf-9ed67e417c56 tempest-VolumesActionsTest-1083035635 None] Created volume successfully. May 28 21:36:05.148684 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-185d8769-1e10-42aa-b760-4b50369dfbe7-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:05.264942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:05.430167 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:36:05.430519 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] Volume volume-185d8769-1e10-42aa-b760-4b50369dfbe7 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:05.676844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] deleting rbd volume volume-185d8769-1e10-42aa-b760-4b50369dfbe7 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:11.305831 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] Created reservations ['df674998-11a2-4d83-9d11-1fcb3ab22b38', 'cc2b8042-16f3-40d8-9847-d9b52b79fc62', '30bdb641-80e1-4bd2-9998-d604c6c7b883', 'b212b98c-d3f0-40df-a784-bdf96e2c4101'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:11.529268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:11.597745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:11.621975 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:11.626484 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] Deleted volume successfully. May 28 21:36:11.789586 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a87f6ef3-672c-4024-9777-258ad54f8dab tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-185d8769-1e10-42aa-b760-4b50369dfbe7-delete_volume'" released by "delete_volume" :: held 6.641s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:12.284068 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:36:12.302273 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (0e71a83e-7206-4c9a-9112-7427d66af67e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:12.309400 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (30cecad3-e9ed-449a-9809-3520b6118330) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:12.327958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (30cecad3-e9ed-449a-9809-3520b6118330) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:36:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-225330235',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=676e69ce-a89e-4ac3-a991-e43a72a585f0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='28f8fcc4afe8471c94db58bef7526459',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:36:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:36:12Z,use_quota=True,user_id='b8372df2880a483e8f1b2b1b667ddf04',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:12.333286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0b78916-e47a-404c-a8dd-2634d4823846) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:12.337467 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f0b78916-e47a-404c-a8dd-2634d4823846) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:12.342731 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (911277f1-14dc-4484-a177-7dfb65ec96e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:12.346727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (911277f1-14dc-4484-a177-7dfb65ec96e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '676e69ce-a89e-4ac3-a991-e43a72a585f0', 'volume_name': 'volume-676e69ce-a89e-4ac3-a991-e43a72a585f0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:12.351566 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4b3db1c5-ad62-4294-8d63-58a2bde18f55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:12.355432 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4b3db1c5-ad62-4294-8d63-58a2bde18f55) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:12.360089 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9a2a6ca6-c0cf-4557-87aa-98d730599988) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:12.361140 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Volume 676e69ce-a89e-4ac3-a991-e43a72a585f0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-676e69ce-a89e-4ac3-a991-e43a72a585f0', 'volume_size': 1} May 28 21:36:12.361524 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] creating volume 'volume-676e69ce-a89e-4ac3-a991-e43a72a585f0' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:36:12.362308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:13.155285 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:36:13.155855 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:36:13.159789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9a2a6ca6-c0cf-4557-87aa-98d730599988) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-676e69ce-a89e-4ac3-a991-e43a72a585f0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:13.164569 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (534b088c-4022-48ee-ade7-a4b50a5aa335) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:13.169606 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=53,cinder:UPDATE=21,cinder:INSERT=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:13.281416 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Volume volume-676e69ce-a89e-4ac3-a991-e43a72a585f0 (676e69ce-a89e-4ac3-a991-e43a72a585f0): created successfully May 28 21:36:13.285189 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (534b088c-4022-48ee-ade7-a4b50a5aa335) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:13.290709 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (0e71a83e-7206-4c9a-9112-7427d66af67e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:13.388573 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f8c09da3-693a-4d9a-9700-47efcadc865e tempest-VolumesActionsTest-1083035635 None] Created volume successfully. May 28 21:36:14.543735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-676e69ce-a89e-4ac3-a991-e43a72a585f0-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:14.572753 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-b1db5f50-dee7-439c-a15e-8e9ff123720f req-64dc57a7-7eca-4666-bccf-00b0d1f65fba tempest-AttachVolumeNegativeTest-679324601 None] Terminate volume connection completed successfully. May 28 21:36:14.573114 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-b1db5f50-dee7-439c-a15e-8e9ff123720f req-64dc57a7-7eca-4666-bccf-00b0d1f65fba tempest-AttachVolumeNegativeTest-679324601 None] Deleting attachment f726fa80-2a19-4ce9-a14e-d240d214734c. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:36:14.661017 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:14.812354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:36:14.812717 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] Volume volume-676e69ce-a89e-4ac3-a991-e43a72a585f0 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:14.984220 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] deleting rbd volume volume-676e69ce-a89e-4ac3-a991-e43a72a585f0 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:20.163119 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] Created reservations ['f7d3694f-c6ce-4ea0-ba31-61aace3b7118', 'a4de04f8-d9d3-4299-8e96-1a983b90cc64', '0737dc81-8c1b-4d76-aded-50fe0d72156e', '995e8fff-576f-4915-818d-6c5c483a0ff9'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:20.344386 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:20.365370 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:20.388365 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:20.392283 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] Deleted volume successfully. May 28 21:36:20.463889 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ba15eedd-f0c0-4f9e-9ddb-6c0ea7cd2ed1 tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-676e69ce-a89e-4ac3-a991-e43a72a585f0-delete_volume'" released by "delete_volume" :: held 5.921s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:21.981641 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:36:22.000489 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (766253a0-0bbc-41a9-8c5f-b98d661ba094) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:22.006810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f96d21c3-2649-4718-b58c-7faec5e4720b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:22.024104 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f96d21c3-2649-4718-b58c-7faec5e4720b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:36:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-15226533',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=a6e387f0-38da-4cfa-838c-97090e72aaa3,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='28f8fcc4afe8471c94db58bef7526459',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:36:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:36:22Z,use_quota=True,user_id='b8372df2880a483e8f1b2b1b667ddf04',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:22.030076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (218264e2-9657-4596-a641-ed71ca537b9b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:22.034160 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (218264e2-9657-4596-a641-ed71ca537b9b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:22.039187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3e632326-42c8-4cd5-b3ec-288e9b9a0772) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:22.043720 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3e632326-42c8-4cd5-b3ec-288e9b9a0772) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a6e387f0-38da-4cfa-838c-97090e72aaa3', 'volume_name': 'volume-a6e387f0-38da-4cfa-838c-97090e72aaa3', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:22.048591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a78d8813-18e8-4c2e-b777-bd427abde6cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:22.052477 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a78d8813-18e8-4c2e-b777-bd427abde6cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:22.057113 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cea799f7-1b84-4684-ac15-f1f274291d2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:22.058229 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Volume a6e387f0-38da-4cfa-838c-97090e72aaa3: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a6e387f0-38da-4cfa-838c-97090e72aaa3', 'volume_size': 1} May 28 21:36:22.058612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] creating volume 'volume-a6e387f0-38da-4cfa-838c-97090e72aaa3' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:36:22.059355 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:22.857482 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:36:22.857965 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:36:22.861998 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cea799f7-1b84-4684-ac15-f1f274291d2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a6e387f0-38da-4cfa-838c-97090e72aaa3', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:22.866813 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4f4366d3-98f0-422a-86e4-6a0cbda4d1db) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:22.871721 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=61,cinder:UPDATE=29,cinder:DELETE=5,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:22.998552 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Volume volume-a6e387f0-38da-4cfa-838c-97090e72aaa3 (a6e387f0-38da-4cfa-838c-97090e72aaa3): created successfully May 28 21:36:23.002973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4f4366d3-98f0-422a-86e4-6a0cbda4d1db) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:23.008479 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (766253a0-0bbc-41a9-8c5f-b98d661ba094) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:23.111683 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-23276853-2fac-45d0-8bb2-84025ba509bd tempest-VolumesActionsTest-1083035635 None] Created volume successfully. May 28 21:36:24.345555 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-a6e387f0-38da-4cfa-838c-97090e72aaa3-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:24.460704 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:24.723351 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:36:24.723784 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] Volume volume-a6e387f0-38da-4cfa-838c-97090e72aaa3 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:24.944086 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] deleting rbd volume volume-a6e387f0-38da-4cfa-838c-97090e72aaa3 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:26.612264 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-0ecce648-3798-4c5f-9d35-6c463a370ccf req-ef6f932d-a7b3-4a8e-aa33-5eea8638d276 tempest-TestVolumeBackupRestore-1923413318 None] Terminate volume connection completed successfully. May 28 21:36:26.612849 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-0ecce648-3798-4c5f-9d35-6c463a370ccf req-ef6f932d-a7b3-4a8e-aa33-5eea8638d276 tempest-TestVolumeBackupRestore-1923413318 None] Deleting attachment be2c4092-4534-4aee-adba-6c13e90832cc. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:36:27.915747 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] Created reservations ['bf790255-0ea4-48d9-b450-3b78dbca22b9', '18968343-7af4-4932-aa3f-e0ef67a0a9c3', 'b6755147-c10b-49bc-bd81-90c6a52c0ca3', '709edb1f-fbb1-4dca-a394-05ca2dfd636b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:28.140591 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:28.165732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:28.188162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:28.192009 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] Deleted volume successfully. May 28 21:36:28.269305 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-269f8e04-a76f-4fa1-9c00-adde6dd4d006 tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-a6e387f0-38da-4cfa-838c-97090e72aaa3-delete_volume'" released by "delete_volume" :: held 3.924s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:28.304286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] Lock "b'cinder-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:28.419677 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:30.809403 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] Volume volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:31.134014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] deleting rbd volume volume-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:31.159623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] Lock "b'cinder-bac5fe86-e590-414b-bff3-906130d40c5b-delete_volume'" acquired by "delete_volume" :: waited 0.019s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:31.238575 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:36:31.259153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (bc97afdc-cbce-43b1-a1c2-a680b02763a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:31.267040 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (799b30d2-5b19-4cf5-965c-24dc1550b374) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:31.286325 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (799b30d2-5b19-4cf5-965c-24dc1550b374) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:36:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-48017895',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=6f3f0b9c-1bbf-432e-8791-04dd3439be23,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='28f8fcc4afe8471c94db58bef7526459',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:36:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:36:29Z,use_quota=True,user_id='b8372df2880a483e8f1b2b1b667ddf04',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:31.292138 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (dbab3741-ecfc-4539-be65-c0596dd304ef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:31.296489 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (dbab3741-ecfc-4539-be65-c0596dd304ef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:31.301574 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b4f8d5f9-7165-4f5d-a560-94b99a2909bc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:31.305651 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b4f8d5f9-7165-4f5d-a560-94b99a2909bc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6f3f0b9c-1bbf-432e-8791-04dd3439be23', 'volume_name': 'volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:31.310887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (18b6c4b0-ca9e-43bf-9106-82e8c116b012) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:31.314988 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (18b6c4b0-ca9e-43bf-9106-82e8c116b012) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:31.319807 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (55d32f8b-c945-4541-a3bc-72518e30a372) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:31.320872 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Volume 6f3f0b9c-1bbf-432e-8791-04dd3439be23: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23', 'volume_size': 1} May 28 21:36:31.321265 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] creating volume 'volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:36:31.322089 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:31.378747 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:34.162461 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] deleting rbd volume volume-bac5fe86-e590-414b-bff3-906130d40c5b {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:34.175288 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:36:34.175680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:36:34.179661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (55d32f8b-c945-4541-a3bc-72518e30a372) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:34.185978 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9d69889e-f7dc-4574-9ec1-36f881731719) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:34.265534 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Volume volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23 (6f3f0b9c-1bbf-432e-8791-04dd3439be23): created successfully May 28 21:36:34.270650 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9d69889e-f7dc-4574-9ec1-36f881731719) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:34.276374 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (bc97afdc-cbce-43b1-a1c2-a680b02763a1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:34.288324 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-7572a51b-e644-45e8-a3ff-5780374e8390 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['8abf8977-1be6-4fad-a19c-d5a1fea6cde2', 'b8dfec24-4afc-472a-80db-ed99926016e3', '46857d91-7d5e-47e8-af84-3d5d253169b0', '7d9da75c-154e-4ed1-bc93-a25df9dba03b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:34.357275 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-93b8b334-a334-4367-99b9-2ced34adaabe tempest-VolumesActionsTest-1083035635 None] Created volume successfully. May 28 21:36:34.516737 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-7572a51b-e644-45e8-a3ff-5780374e8390 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:36:34.522125 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7572a51b-e644-45e8-a3ff-5780374e8390 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-b87f39f6-7332-42a8-baeb-0ad2a8a17b41-delete_snapshot'" released by "delete_snapshot" :: held 36.444s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:34.891168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-6f3f0b9c-1bbf-432e-8791-04dd3439be23-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:34.988117 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=65,cinder:UPDATE=27,cinder:INSERT=5,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:34.997239 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:35.255387 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:36:35.255950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] Volume volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:35.451478 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] deleting rbd volume volume-6f3f0b9c-1bbf-432e-8791-04dd3439be23 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:36.399894 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-66c83d98-1794-444c-bc12-a09f10f59649 req-b821fee2-61ba-41c7-a15d-a710439a2a4e tempest-VolumesSnapshotTestJSON-961770574 None] Terminate volume connection completed successfully. May 28 21:36:36.400636 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-66c83d98-1794-444c-bc12-a09f10f59649 req-b821fee2-61ba-41c7-a15d-a710439a2a4e tempest-VolumesSnapshotTestJSON-961770574 None] Deleting attachment 9d7d5980-cfd9-44d3-a11a-a2087fcde8dd. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:36:39.477046 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] Created reservations ['e98b87c9-1e12-4d20-b642-18f46caabdde', '534e09db-0fcb-4942-9f2f-4cb2419f526a', 'ae2392af-3a1d-4803-af84-031649c654da', '997e481f-8471-431a-9aa9-38204ecad3ba'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:39.734802 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:39.783057 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:39.814144 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:39.819058 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] Deleted volume successfully. May 28 21:36:39.872280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:36:40.049166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] deleting parent snapshot volume-bac5fe86-e590-414b-bff3-906130d40c5b.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:36:42.188147 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-80fb600b-08ff-4dd4-a18e-b7ab26d32805 tempest-TestVolumeBackupRestore-1923413318 None] Lock "b'cinder-66ff15ab-0cf7-464f-a399-0ea2ae0a2aa0-delete_volume'" released by "delete_volume" :: held 13.883s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:42.266942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a915968f-a05b-4b7f-97a2-8cf96c5c2327 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:42.386885 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] Created reservations ['4ba4090f-6c1b-4dcf-bd1f-292e2be01afc', '41364c7d-cb77-4c63-8a7d-128e19d69981', 'd438cf27-8c6e-4f20-82ef-dc605e7a725e', '3711b67c-271a-4be2-981a-48a3d64f3ebc'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:42.390180 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] Created reservations ['0acd4789-e5cd-4e18-98e6-ff3b6153a2cb', '2fd6b1f3-79f6-4769-b205-98064282e978', 'c7dfa45c-241a-4c36-a1fb-1baa22c3c8eb', 'da552003-2433-49a9-9bc9-3dac23ef74ce'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:42.482449 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=55,cinder:UPDATE=32,cinder:INSERT=13 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:42.628333 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:42.650116 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:42.673378 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:42.680201 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] Deleted volume successfully. May 28 21:36:42.763919 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:42.785043 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:42.806857 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:42.810959 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] Deleted volume successfully. May 28 21:36:42.814391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-137c3037-6cb7-4185-929c-7d24cc519b54 tempest-TestVolumeBackupRestore-1923413318 None] Lock "b'cinder-bac5fe86-e590-414b-bff3-906130d40c5b-delete_volume'" released by "delete_volume" :: held 11.655s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:42.863521 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-3101641b-6d65-4ea2-acaa-8350e22778ef tempest-VolumesActionsTest-1662161443 None] Lock "b'cinder-6f3f0b9c-1bbf-432e-8791-04dd3439be23-delete_volume'" released by "delete_volume" :: held 7.972s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:43.824023 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a915968f-a05b-4b7f-97a2-8cf96c5c2327 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:36:44.152677 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:36:44.172912 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (2fd6787c-6316-46df-ae80-1160aa8f5df3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:44.179622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ecc6c89c-be95-4ff3-a6da-901a16e4a859) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:44.199018 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ecc6c89c-be95-4ff3-a6da-901a16e4a859) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:36:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesActionsTest-Volume-1940991847',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=81d8fff7-9b49-445e-bf78-100acd2bd6e0,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='28f8fcc4afe8471c94db58bef7526459',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:36:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:36:44Z,use_quota=True,user_id='b8372df2880a483e8f1b2b1b667ddf04',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:44.204409 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0e936bd3-afdd-4477-9607-b558ca24cdba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:44.208629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0e936bd3-afdd-4477-9607-b558ca24cdba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:44.213956 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ee0fb0e-557b-489c-8963-acc745898acc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:44.218355 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3ee0fb0e-557b-489c-8963-acc745898acc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '81d8fff7-9b49-445e-bf78-100acd2bd6e0', 'volume_name': 'volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:44.223877 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4bf5e289-5869-4d71-b07b-b913e34bd810) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:44.227721 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4bf5e289-5869-4d71-b07b-b913e34bd810) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:44.232424 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8b49f67d-a36f-4fa0-a917-15fcc12c5508) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:44.233702 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Volume 81d8fff7-9b49-445e-bf78-100acd2bd6e0: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0', 'volume_size': 1} May 28 21:36:44.233941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] creating volume 'volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:36:44.235357 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:44.733422 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2baed6d4-f810-462c-879a-d21a142833e0 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-36a2dde2-6e31-4a6e-ae11-c21acb6be45f-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:44.735420 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2baed6d4-f810-462c-879a-d21a142833e0 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:45.182623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:36:45.183232 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:36:45.189485 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (8b49f67d-a36f-4fa0-a917-15fcc12c5508) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:45.192100 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (abc22eb1-4efc-45ab-91c6-a68c9f827dd0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:36:45.265429 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Volume volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0 (81d8fff7-9b49-445e-bf78-100acd2bd6e0): created successfully May 28 21:36:45.269608 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (abc22eb1-4efc-45ab-91c6-a68c9f827dd0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:36:45.275345 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Flow 'volume_create_manager' (2fd6787c-6316-46df-ae80-1160aa8f5df3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:36:45.361481 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fe33e410-f359-4fc5-99e1-c00cc8f160b9 tempest-VolumesActionsTest-1083035635 None] Created volume successfully. May 28 21:36:47.018044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-81d8fff7-9b49-445e-bf78-100acd2bd6e0-delete_volume'" acquired by "delete_volume" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:47.132626 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:47.270787 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:36:47.271225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] Volume volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:47.425399 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] deleting rbd volume volume-81d8fff7-9b49-445e-bf78-100acd2bd6e0 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:47.561118 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2baed6d4-f810-462c-879a-d21a142833e0 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['06eebf73-ca41-4acf-8780-c3bc87d9e6b3', '3d52ea9f-5eff-45c3-a83d-7f18d80c3b11', '8c0bf89e-c414-418d-960c-26f82d1ba59f', '478f2db6-3fbb-469c-85ef-c8ea21080d12'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:47.691007 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=52,cinder:UPDATE=38,cinder:DELETE=4,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:36:47.797248 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2baed6d4-f810-462c-879a-d21a142833e0 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:36:47.802982 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2baed6d4-f810-462c-879a-d21a142833e0 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-36a2dde2-6e31-4a6e-ae11-c21acb6be45f-delete_snapshot'" released by "delete_snapshot" :: held 3.070s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:48.351012 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-eecedfe0-e969-460b-bbb1-5767aa692cc2 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:51.134291 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-eecedfe0-e969-460b-bbb1-5767aa692cc2 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:36:51.266090 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] Created reservations ['752fd099-72e2-4634-91c6-862a4857d796', '7d8af1fc-5a17-4184-9884-23ffac12f098', 'd500c930-dc0a-4ce9-a2cc-a6f2748274c8', '8f72aeb5-c8b5-4795-ad04-70c00d08483d'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:51.506192 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:51.540713 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:51.588664 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:51.594089 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] Deleted volume successfully. May 28 21:36:51.655106 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-4384f5e0-9149-4773-9673-b4a6d5b9966d tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-81d8fff7-9b49-445e-bf78-100acd2bd6e0-delete_volume'" released by "delete_volume" :: held 4.637s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:36:52.607686 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:36:52.717799 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:52.836401 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:36:52.836833 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] Volume volume-ec5a0831-5a82-4255-821f-f73fb72a98be is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:36:52.888389 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] deleting rbd volume volume-ec5a0831-5a82-4255-821f-f73fb72a98be {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:36:55.606467 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] Created reservations ['2ae2530b-3561-4760-86d8-15189b999941', 'da66ab65-77f3-4cf1-a1b8-e4c761acff09', 'eb28c7b2-54c4-4c46-8fd9-466f2ebaa7b4', '4b5548da-b78d-4355-a9c5-6330fbe6c35b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:36:55.786437 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:55.824404 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:36:55.873270 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:36:55.878078 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] Deleted volume successfully. May 28 21:36:55.949954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-cadd8dca-bb71-4430-84bd-5f8adb9182b6 tempest-VolumesActionsTest-1083035635 None] Lock "b'cinder-ec5a0831-5a82-4255-821f-f73fb72a98be-delete_volume'" released by "delete_volume" :: held 3.342s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:37:01.082034 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:37:01.082763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:01.105922 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:01.151224 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:37:03.095072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:37:03.114602 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Flow 'volume_create_manager' (2ba03037-d3c6-4b5c-9922-d459beb104c9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:03.121111 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96ace76a-e143-4991-9e7f-06096c2761ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:03.136964 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (96ace76a-e143-4991-9e7f-06096c2761ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:37:03Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotMetadataTestJSON-Volume-1215625555',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=98c89e67-7cd1-4699-8141-6bfbc2032d00,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9b50da35b3424021ad71774889dabb2d',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:37:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:37:03Z,use_quota=True,user_id='5b10fdc16ad64679be36d3cb907fed0a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:03.142416 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f77d8d97-23e0-42ac-9797-f1575ebbd424) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:03.147111 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f77d8d97-23e0-42ac-9797-f1575ebbd424) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:03.152183 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08dff31c-c578-4f25-a83a-58cea0d6f44d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:03.156187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08dff31c-c578-4f25-a83a-58cea0d6f44d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '98c89e67-7cd1-4699-8141-6bfbc2032d00', 'volume_name': 'volume-98c89e67-7cd1-4699-8141-6bfbc2032d00', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:03.161041 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (46166da9-a572-4dd2-be82-ab7475facabc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:03.164899 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (46166da9-a572-4dd2-be82-ab7475facabc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:03.169604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05954544-be66-4e49-8a24-8f19d0fce8ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:03.170690 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Volume 98c89e67-7cd1-4699-8141-6bfbc2032d00: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-98c89e67-7cd1-4699-8141-6bfbc2032d00', 'volume_size': 1} May 28 21:37:03.171075 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] creating volume 'volume-98c89e67-7cd1-4699-8141-6bfbc2032d00' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:37:03.171816 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:03.742440 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:37:03.742819 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:37:03.746616 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (05954544-be66-4e49-8a24-8f19d0fce8ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-98c89e67-7cd1-4699-8141-6bfbc2032d00', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:03.751339 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2ddea6c4-1db9-4ea1-8eff-5d6e3b2baafd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:03.808119 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Volume volume-98c89e67-7cd1-4699-8141-6bfbc2032d00 (98c89e67-7cd1-4699-8141-6bfbc2032d00): created successfully May 28 21:37:03.811845 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2ddea6c4-1db9-4ea1-8eff-5d6e3b2baafd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:03.817979 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Flow 'volume_create_manager' (2ba03037-d3c6-4b5c-9922-d459beb104c9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:03.899778 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d0f7c9d5-adf8-4217-8a63-86ec2a70a5d2 tempest-SnapshotMetadataTestJSON-1521779861 None] Created volume successfully. May 28 21:37:03.903413 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=56,cinder:UPDATE=34,cinder:DELETE=2,cinder:INSERT=8 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:04.757689 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d8efde6b-4e04-4ff3-bf45-a6e24a9754ab tempest-SnapshotMetadataTestJSON-1521779861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:04.932343 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-a7511edd-d36b-4d99-b670-85a3bab7594f req-2108ff41-f72c-4402-bd78-718750015e70 tempest-VolumesSnapshotTestJSON-961770574 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:37:05.546888 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-a7511edd-d36b-4d99-b670-85a3bab7594f req-2108ff41-f72c-4402-bd78-718750015e70 tempest-VolumesSnapshotTestJSON-961770574 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.614s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:37:05.547700 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-a7511edd-d36b-4d99-b670-85a3bab7594f req-2108ff41-f72c-4402-bd78-718750015e70 tempest-VolumesSnapshotTestJSON-961770574 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b31650fd-bab6-499a-bca7-af6b551f165e', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'b31650fd-bab6-499a-bca7-af6b551f165e', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:37:05.651950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-a7511edd-d36b-4d99-b670-85a3bab7594f req-2108ff41-f72c-4402-bd78-718750015e70 tempest-VolumesSnapshotTestJSON-961770574 None] Connection info returned from driver {'name': 'volumes/volume-b31650fd-bab6-499a-bca7-af6b551f165e', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b31650fd-bab6-499a-bca7-af6b551f165e', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ae758bd8-cdac-42a8-a96a-a4d23c3cbcda'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:37:05.782908 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-a7511edd-d36b-4d99-b670-85a3bab7594f req-2108ff41-f72c-4402-bd78-718750015e70 tempest-VolumesSnapshotTestJSON-961770574 None] attachment_update completed successfully. May 28 21:37:06.989361 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d8efde6b-4e04-4ff3-bf45-a6e24a9754ab tempest-SnapshotMetadataTestJSON-1521779861 None] Create snapshot completed successfully May 28 21:37:08.520789 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-00fdc2ec-092c-407c-8c6c-5cfe1fb8e3af tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:08.932263 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ae836726-b754-4335-8782-481dea7b8f81 tempest-SnapshotMetadataTestJSON-1521779861 None] Lock "b'cinder-ed88b2c8-3011-4996-8054-0335395c9bf1-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:37:08.933687 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ae836726-b754-4335-8782-481dea7b8f81 tempest-SnapshotMetadataTestJSON-1521779861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:10.207637 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-00fdc2ec-092c-407c-8c6c-5cfe1fb8e3af tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:37:10.960012 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-8660c8f7-7773-4f3a-be6b-146af11dce76 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:11.573474 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-ae836726-b754-4335-8782-481dea7b8f81 tempest-SnapshotMetadataTestJSON-1521779861 None] Created reservations ['5e0cb166-d51b-4d11-b4f9-15a277e1ebb8', 'cce0fddb-ca2d-45a5-9e76-30e5e512f0aa', 'd5608870-8fdb-4fe9-b0d3-4e74277cd1a5', 'd69c7071-8752-4a71-a099-1c2cde2c041c'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:37:11.666312 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=69,cinder:DELETE=3,cinder:INSERT=3,cinder:UPDATE=25 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:11.749574 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ae836726-b754-4335-8782-481dea7b8f81 tempest-SnapshotMetadataTestJSON-1521779861 None] Delete snapshot completed successfully. May 28 21:37:11.754015 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ae836726-b754-4335-8782-481dea7b8f81 tempest-SnapshotMetadataTestJSON-1521779861 None] Lock "b'cinder-ed88b2c8-3011-4996-8054-0335395c9bf1-delete_snapshot'" released by "delete_snapshot" :: held 2.822s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:37:12.233655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] Lock "b'cinder-98c89e67-7cd1-4699-8141-6bfbc2032d00-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:37:12.352891 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:12.487655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:37:12.488051 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] Volume volume-98c89e67-7cd1-4699-8141-6bfbc2032d00 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:37:12.539329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] deleting rbd volume volume-98c89e67-7cd1-4699-8141-6bfbc2032d00 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:37:12.762431 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-8660c8f7-7773-4f3a-be6b-146af11dce76 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:37:12.956510 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e4474078-cdc6-4eef-9a8b-345ebef5ff5f tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-6c4f2fef-9204-4f3a-a01f-b2f290df870c-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:37:12.958139 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e4474078-cdc6-4eef-9a8b-345ebef5ff5f tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:14.228040 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:37:14.248294 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Flow 'volume_create_manager' (397d8797-d134-4700-9cb6-958df55128a0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:14.254758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (38533cb3-aafd-4292-bb52-b9fbaa00b5e1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:14.270026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (38533cb3-aafd-4292-bb52-b9fbaa00b5e1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:37:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-823034828',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=288fd4fc-4ae9-43dc-9f8e-7c3076f69091,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c9caba58394e1480acb632405111c5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:37:14Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:37:14Z,use_quota=True,user_id='b16d0e791edd4f119e646e7ced74bf74',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:14.275292 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (36c54e7b-f3c4-4313-ab47-399b78cd2b8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:14.279467 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (36c54e7b-f3c4-4313-ab47-399b78cd2b8a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:14.284503 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3b075e0f-36eb-494e-a8f1-a22b14b382e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:14.288498 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3b075e0f-36eb-494e-a8f1-a22b14b382e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'volume_name': 'volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:14.293856 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6e9721ef-2794-4dec-8dc9-aa7a3dcf9a35) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:14.297704 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6e9721ef-2794-4dec-8dc9-aa7a3dcf9a35) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:14.302431 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0fbffc8d-1a0c-4f36-ae80-f32bbe28dea3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:14.303503 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Volume 288fd4fc-4ae9-43dc-9f8e-7c3076f69091: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'volume_size': 1} May 28 21:37:14.303898 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] creating volume 'volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:37:14.304655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:15.203430 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:37:15.203728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:37:15.207426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0fbffc8d-1a0c-4f36-ae80-f32bbe28dea3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:15.212728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b23b0b0d-590f-4ba8-b7c3-6ba90090212e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:15.316352 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Volume volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091 (288fd4fc-4ae9-43dc-9f8e-7c3076f69091): created successfully May 28 21:37:15.320026 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b23b0b0d-590f-4ba8-b7c3-6ba90090212e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:15.326158 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Flow 'volume_create_manager' (397d8797-d134-4700-9cb6-958df55128a0) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:15.440780 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5e7df8e3-ffaa-4c2c-bef9-a47d74347617 tempest-AttachVolumeNegativeTest-679324601 None] Created volume successfully. May 28 21:37:15.899987 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] Created reservations ['57b2d2af-cb15-473a-9318-9046a72e647b', 'a384ac8e-cd35-43d4-9e3f-76ccef271560', '353adf83-e5a5-457e-a13c-255788d49d38', '11105a0c-f72c-483d-8c72-c3e2d874552e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:37:16.134281 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:16.163432 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:16.186085 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:37:16.193729 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] Deleted volume successfully. May 28 21:37:16.273361 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e2098bb0-32d4-4317-853e-0878567b7366 tempest-SnapshotMetadataTestJSON-1521779861 None] Lock "b'cinder-98c89e67-7cd1-4699-8141-6bfbc2032d00-delete_volume'" released by "delete_volume" :: held 4.040s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:37:17.065299 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-5b18aac4-593a-46fe-991c-3487e2384356 req-eeb5089c-5cf1-4ee7-a9d4-5574d3185bc5 tempest-AttachVolumeNegativeTest-679324601 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:37:17.786947 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-5b18aac4-593a-46fe-991c-3487e2384356 req-eeb5089c-5cf1-4ee7-a9d4-5574d3185bc5 tempest-AttachVolumeNegativeTest-679324601 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.722s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:37:17.787798 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-5b18aac4-593a-46fe-991c-3487e2384356 req-eeb5089c-5cf1-4ee7-a9d4-5574d3185bc5 tempest-AttachVolumeNegativeTest-679324601 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:37:17.928554 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-5b18aac4-593a-46fe-991c-3487e2384356 req-eeb5089c-5cf1-4ee7-a9d4-5574d3185bc5 tempest-AttachVolumeNegativeTest-679324601 None] Connection info returned from driver {'name': 'volumes/volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '288fd4fc-4ae9-43dc-9f8e-7c3076f69091', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c0de7be2-bfb8-4b5b-901f-9cd132eda666'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:37:18.067625 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-5b18aac4-593a-46fe-991c-3487e2384356 req-eeb5089c-5cf1-4ee7-a9d4-5574d3185bc5 tempest-AttachVolumeNegativeTest-679324601 None] attachment_update completed successfully. May 28 21:37:21.156493 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:37:21.177451 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Flow 'volume_create_manager' (f1c124c0-e62f-4fde-a486-b42e80be28bd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:21.187740 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (348eae87-8155-420b-a696-83941f1335fd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:21.199891 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (348eae87-8155-420b-a696-83941f1335fd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:37:21Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-1042986051',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=5b6b7258-1904-4203-ab9d-c9e7a5dbc324,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2d4864de54c14fad83350fc6c0854158',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:37:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:37:21Z,use_quota=True,user_id='8dfd178b4a654597a524be7ff2e86db5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:21.205137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d25ce342-1118-456f-b5a1-8c99e8402c1a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:21.209225 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d25ce342-1118-456f-b5a1-8c99e8402c1a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:21.215092 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db6d61a3-50b9-41af-85a1-6224b88cdee9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:21.219098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (db6d61a3-50b9-41af-85a1-6224b88cdee9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5b6b7258-1904-4203-ab9d-c9e7a5dbc324', 'volume_name': 'volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:21.223956 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b43c65ce-54ff-4f94-b551-8c7bd49674a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:21.228404 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b43c65ce-54ff-4f94-b551-8c7bd49674a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:21.233038 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (57eaa8d2-24dd-4571-923c-b176801609be) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:21.234113 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Volume 5b6b7258-1904-4203-ab9d-c9e7a5dbc324: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324', 'volume_size': 1} May 28 21:37:21.234516 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] creating volume 'volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:37:21.235466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:21.318677 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-91990afa-8ed1-4cc6-845e-6e75e5df2b14 req-43c10994-5e48-4766-a1dd-b630223514e5 tempest-AttachVolumeNegativeTest-679324601 None] Terminate volume connection completed successfully. May 28 21:37:21.319078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-91990afa-8ed1-4cc6-845e-6e75e5df2b14 req-43c10994-5e48-4766-a1dd-b630223514e5 tempest-AttachVolumeNegativeTest-679324601 None] Deleting attachment c0de7be2-bfb8-4b5b-901f-9cd132eda666. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:37:21.885137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:37:21.885516 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:37:21.889424 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (57eaa8d2-24dd-4571-923c-b176801609be) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:21.894037 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4a475eb8-aac6-416f-a19a-3a1a515a3e3c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:21.900224 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=61,cinder:UPDATE=29,cinder:DELETE=5,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:21.996492 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Volume volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324 (5b6b7258-1904-4203-ab9d-c9e7a5dbc324): created successfully May 28 21:37:22.001185 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (4a475eb8-aac6-416f-a19a-3a1a515a3e3c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:22.007870 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Flow 'volume_create_manager' (f1c124c0-e62f-4fde-a486-b42e80be28bd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:22.092269 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d857d3df-c3d1-4808-ae70-9afba7b66fa1 tempest-VolumesTransfersTest-1312661495 None] Created volume successfully. May 28 21:37:24.171069 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-aaf77cd0-1d07-40ff-b69c-062582d3669a tempest-VolumesTransfersTest-402381136 None] Transfer volume completed successfully. May 28 21:37:25.367058 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] Lock "b'cinder-5b6b7258-1904-4203-ab9d-c9e7a5dbc324-delete_volume'" acquired by "delete_volume" :: waited 0.021s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:37:25.474607 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:25.579121 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:37:25.579559 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] Volume volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:37:25.658740 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] deleting rbd volume volume-5b6b7258-1904-4203-ab9d-c9e7a5dbc324 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:37:28.285406 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] Created reservations ['4bc01e8f-f74f-46fd-8f9e-292f94458de4', '8c63d4ce-d881-4a0c-9573-74c9643ec6db', '5e341a99-131f-475b-b8c3-d1ba49a163d4', '103b18e2-d894-4c42-b82c-66fc6f2a911b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:37:28.449002 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:28.477152 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:28.501803 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:37:28.508123 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] Deleted volume successfully. May 28 21:37:28.612577 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-fb7ec9c7-0ab8-4fd2-b171-160c67caaf02 tempest-VolumesTransfersTest-1259230223 None] Lock "b'cinder-5b6b7258-1904-4203-ab9d-c9e7a5dbc324-delete_volume'" released by "delete_volume" :: held 3.246s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:37:29.386129 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:37:29.405800 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Flow 'volume_create_manager' (cc3b4469-ce53-4da4-a02c-38422ff0295d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:29.412795 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (75ce0374-4131-4d66-a94d-7988a2e279a4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:29.428858 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (75ce0374-4131-4d66-a94d-7988a2e279a4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:37:29Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersTest-Volume-352827437',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=decca9af-ccd3-4bb5-9130-9e715ea02760,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='2d4864de54c14fad83350fc6c0854158',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:37:29Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:37:29Z,use_quota=True,user_id='8dfd178b4a654597a524be7ff2e86db5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:29.434187 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7fd7730d-5de1-470b-9786-a667a368f7d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:29.438271 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7fd7730d-5de1-470b-9786-a667a368f7d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:29.443303 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f9bb25b8-4ff7-42cd-8f75-751e75ca2fde) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:29.447338 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f9bb25b8-4ff7-42cd-8f75-751e75ca2fde) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'decca9af-ccd3-4bb5-9130-9e715ea02760', 'volume_name': 'volume-decca9af-ccd3-4bb5-9130-9e715ea02760', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:29.452257 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2e8aeae8-3e32-4c81-aeba-cfc80524cc8e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:29.456132 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (2e8aeae8-3e32-4c81-aeba-cfc80524cc8e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:29.460793 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (69a86dbf-a30f-4c19-af74-147ec3376c24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:29.461884 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Volume decca9af-ccd3-4bb5-9130-9e715ea02760: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-decca9af-ccd3-4bb5-9130-9e715ea02760', 'volume_size': 1} May 28 21:37:29.462277 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] creating volume 'volume-decca9af-ccd3-4bb5-9130-9e715ea02760' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:37:29.463537 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:29.830021 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:37:29.831524 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:37:29.840331 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (69a86dbf-a30f-4c19-af74-147ec3376c24) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-decca9af-ccd3-4bb5-9130-9e715ea02760', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:29.847494 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a63a6864-63d5-45a3-9ec8-7cd0664d8dec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:37:29.951122 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Volume volume-decca9af-ccd3-4bb5-9130-9e715ea02760 (decca9af-ccd3-4bb5-9130-9e715ea02760): created successfully May 28 21:37:29.954792 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a63a6864-63d5-45a3-9ec8-7cd0664d8dec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:37:29.960368 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Flow 'volume_create_manager' (cc3b4469-ce53-4da4-a02c-38422ff0295d) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:37:30.030641 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e89fca16-1721-41c6-b36c-99955ba2a40c tempest-VolumesTransfersTest-1312661495 None] Created volume successfully. May 28 21:37:31.079562 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] Lock "b'cinder-decca9af-ccd3-4bb5-9130-9e715ea02760-delete_volume'" acquired by "delete_volume" :: waited 0.008s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:37:31.168949 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:31.293030 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:37:31.293483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] Volume volume-decca9af-ccd3-4bb5-9130-9e715ea02760 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:37:31.386088 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] deleting rbd volume volume-decca9af-ccd3-4bb5-9130-9e715ea02760 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:37:34.173448 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] Created reservations ['16515279-2d8a-4ee2-9f12-5c193aeb880d', '271a8580-d110-4114-b084-c01bf32a7524', 'f6e8e47d-98cc-40ea-abf4-7d01cff2f8f8', '6a9ea9d7-6dca-4e42-aef6-dc1017bb0411'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:37:34.189364 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=67,cinder:UPDATE=24,cinder:INSERT=7,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:37:34.374245 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:34.398262 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:34.423421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:37:34.428254 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] Deleted volume successfully. May 28 21:37:34.539189 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-debf6c4b-d24f-4e45-be03-164e428d46a6 tempest-VolumesTransfersTest-1259230223 None] Lock "b'cinder-decca9af-ccd3-4bb5-9130-9e715ea02760-delete_volume'" released by "delete_volume" :: held 3.460s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:37:43.893646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e4474078-cdc6-4eef-9a8b-345ebef5ff5f tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['e71b7e52-132a-40c2-97d2-4451722bd805', '089d5f94-e287-493f-ae28-05fa7a3b6984', '39b772b3-73c0-4b11-94cd-9dded057af32', '3555176d-4192-4877-9b61-ad42c20527c4'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:37:44.090530 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e4474078-cdc6-4eef-9a8b-345ebef5ff5f tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:37:44.098350 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e4474078-cdc6-4eef-9a8b-345ebef5ff5f tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-6c4f2fef-9204-4f3a-a01f-b2f290df870c-delete_snapshot'" released by "delete_snapshot" :: held 31.142s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:37:45.059129 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e18bdf6a-847b-43a8-9b74-b3a17eb5f185 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-588f3995-15ca-4f47-94f7-5cd2d727160b-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.020s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:37:45.062275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e18bdf6a-847b-43a8-9b74-b3a17eb5f185 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:37:54.200597 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=48,cinder:UPDATE=31,cinder:INSERT=5,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:01.086198 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:38:01.086970 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:01.111163 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:01.133653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:38:14.204140 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=12,cinder:INSERT=2,cinder:UPDATE=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:15.805986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e18bdf6a-847b-43a8-9b74-b3a17eb5f185 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['ebd71a54-885d-4fb2-822c-83d7d76e1e66', '1087b308-3615-47bf-ab77-756b6c05ae05', '6e58f5ab-fa3c-485b-9403-5fad2113b647', '22305f57-3efb-491b-a7de-bbd130f83527'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:38:15.989129 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e18bdf6a-847b-43a8-9b74-b3a17eb5f185 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:38:15.994653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e18bdf6a-847b-43a8-9b74-b3a17eb5f185 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-588f3995-15ca-4f47-94f7-5cd2d727160b-delete_snapshot'" released by "delete_snapshot" :: held 30.936s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:38:16.121492 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9859ae91-d651-4035-a8f2-0f853d20099c tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-eb6b2fe0-bcf8-421c-b29a-2fb697798146-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:38:16.126248 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9859ae91-d651-4035-a8f2-0f853d20099c tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:16.383138 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:38:16.403900 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Flow 'volume_create_manager' (55e252e3-4e71-48d1-b2d5-f0ee3c0778d5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:38:16.410476 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (75ea574f-f238-43f5-b47a-8c97f4e1ceb9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:16.427920 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (75ea574f-f238-43f5-b47a-8c97f4e1ceb9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:38:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeNegativeTest-volume-682659967',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=8424c958-02ac-4e4c-9145-b75c712c7d07,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f9c9caba58394e1480acb632405111c5',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:38:16Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:38:16Z,use_quota=True,user_id='b16d0e791edd4f119e646e7ced74bf74',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:16.434835 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7725d19d-e9d4-4de0-817e-4d9b670f3704) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:16.439050 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7725d19d-e9d4-4de0-817e-4d9b670f3704) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:16.444837 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7915ab9-3625-49fc-9be2-ef2f10685816) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:16.449754 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f7915ab9-3625-49fc-9be2-ef2f10685816) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '8424c958-02ac-4e4c-9145-b75c712c7d07', 'volume_name': 'volume-8424c958-02ac-4e4c-9145-b75c712c7d07', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:16.454658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (379570d6-9678-4c6b-b8e0-e6d289139e71) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:16.459728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (379570d6-9678-4c6b-b8e0-e6d289139e71) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:16.464826 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dbef9fbb-bb69-44a4-b45a-58abccd22fbd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:16.465922 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Volume 8424c958-02ac-4e4c-9145-b75c712c7d07: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-8424c958-02ac-4e4c-9145-b75c712c7d07', 'volume_size': 1} May 28 21:38:16.466308 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] creating volume 'volume-8424c958-02ac-4e4c-9145-b75c712c7d07' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:38:16.467063 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:17.199428 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:38:17.200035 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:38:17.203535 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dbef9fbb-bb69-44a4-b45a-58abccd22fbd) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-8424c958-02ac-4e4c-9145-b75c712c7d07', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:17.208151 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ee405df-7bb6-4f70-b245-77dcb0e18068) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:17.310220 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Volume volume-8424c958-02ac-4e4c-9145-b75c712c7d07 (8424c958-02ac-4e4c-9145-b75c712c7d07): created successfully May 28 21:38:17.313829 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0ee405df-7bb6-4f70-b245-77dcb0e18068) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:17.319529 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Flow 'volume_create_manager' (55e252e3-4e71-48d1-b2d5-f0ee3c0778d5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:38:17.401668 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a260cb9a-9282-4829-bf9e-7141e64f8cac tempest-AttachVolumeNegativeTest-679324601 None] Created volume successfully. May 28 21:38:19.208299 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-0a95faa9-043a-4f32-a726-2e56514e106b req-38f3c0ec-5257-4739-b0d1-6fb2e3a3b5e4 tempest-AttachVolumeNegativeTest-679324601 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:38:19.832818 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-0a95faa9-043a-4f32-a726-2e56514e106b req-38f3c0ec-5257-4739-b0d1-6fb2e3a3b5e4 tempest-AttachVolumeNegativeTest-679324601 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.624s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:38:19.833658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-0a95faa9-043a-4f32-a726-2e56514e106b req-38f3c0ec-5257-4739-b0d1-6fb2e3a3b5e4 tempest-AttachVolumeNegativeTest-679324601 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-8424c958-02ac-4e4c-9145-b75c712c7d07', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '8424c958-02ac-4e4c-9145-b75c712c7d07', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:38:19.925816 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-0a95faa9-043a-4f32-a726-2e56514e106b req-38f3c0ec-5257-4739-b0d1-6fb2e3a3b5e4 tempest-AttachVolumeNegativeTest-679324601 None] Connection info returned from driver {'name': 'volumes/volume-8424c958-02ac-4e4c-9145-b75c712c7d07', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '8424c958-02ac-4e4c-9145-b75c712c7d07', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'af5dd682-e430-4d38-a29c-0c14f4766f40'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:38:20.030548 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-0a95faa9-043a-4f32-a726-2e56514e106b req-38f3c0ec-5257-4739-b0d1-6fb2e3a3b5e4 tempest-AttachVolumeNegativeTest-679324601 None] attachment_update completed successfully. May 28 21:38:24.432456 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2085482f-c36d-4555-9e83-279016d86610 req-dc2875fa-746e-4993-8b07-8e82bed6f490 tempest-AttachVolumeNegativeTest-679324601 None] Terminate volume connection completed successfully. May 28 21:38:24.433147 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2085482f-c36d-4555-9e83-279016d86610 req-dc2875fa-746e-4993-8b07-8e82bed6f490 tempest-AttachVolumeNegativeTest-679324601 None] Deleting attachment af5dd682-e430-4d38-a29c-0c14f4766f40. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:38:31.950793 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] Lock "b'cinder-8424c958-02ac-4e4c-9145-b75c712c7d07-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:38:31.953488 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=32,cinder:UPDATE=14,cinder:INSERT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:32.075917 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:32.149862 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:38:32.150413 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] Volume volume-8424c958-02ac-4e4c-9145-b75c712c7d07 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:38:32.240206 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] deleting rbd volume volume-8424c958-02ac-4e4c-9145-b75c712c7d07 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:38:34.424174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] Created reservations ['0e788ab3-ea7c-421b-a8ef-b2ac430fc5d1', '1a3ef2c4-8c1f-4654-bcf6-d16d2bd1d819', 'bef158e9-51ad-4c69-9690-1974f14e24ae', 'ffce573f-bec0-4f32-b045-fda3ef3adb3d'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:38:34.539126 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:34.561758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:34.584780 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:38:34.589409 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] Deleted volume successfully. May 28 21:38:34.645951 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9e817eb3-f50b-480f-8085-5320f6b62a05 tempest-AttachVolumeNegativeTest-679324601 None] Lock "b'cinder-8424c958-02ac-4e4c-9145-b75c712c7d07-delete_volume'" released by "delete_volume" :: held 2.695s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:38:35.406008 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] Lock "b'cinder-288fd4fc-4ae9-43dc-9f8e-7c3076f69091-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:38:35.492899 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:35.666994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:38:35.666994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] Volume volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:38:35.807149 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] deleting rbd volume volume-288fd4fc-4ae9-43dc-9f8e-7c3076f69091 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:38:38.519836 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] Created reservations ['63d02e9a-7e8f-4135-98c4-0e7a02e1b5fa', '17153314-ccb9-4acc-aabb-d144b0542af5', '56f0e7c1-5392-402d-8f91-f7783e2832b8', '71741719-4742-46f7-9ef4-d907dc2792be'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:38:38.671968 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:38.697757 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:38.721648 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:38:38.725766 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] Deleted volume successfully. May 28 21:38:38.800191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1d5bce0e-bcce-499d-929b-065ecbf40d4a tempest-AttachVolumeNegativeTest-679324601 None] Lock "b'cinder-288fd4fc-4ae9-43dc-9f8e-7c3076f69091-delete_volume'" released by "delete_volume" :: held 3.394s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:38:39.977592 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] Lock "b'cinder-9ae207c8-8c87-4201-942c-781ab5b960e5-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:38:40.085164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:40.224252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:38:40.224252 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] Volume volume-9ae207c8-8c87-4201-942c-781ab5b960e5 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:38:40.270485 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] deleting rbd volume volume-9ae207c8-8c87-4201-942c-781ab5b960e5 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:38:43.230211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] Created reservations ['a519bc38-b721-42f9-9ba1-07f75a47f8ef', 'e30dafe0-8de4-41dc-98c5-bad39ecc46a4', '9fb9cdef-63d3-4945-8705-da0d1d481e95', '429aba64-cb1a-4b53-ac96-2e195c4ca6ae'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:38:43.256472 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=65,cinder:INSERT=9,cinder:UPDATE=24,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:43.486782 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:43.514628 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:43.566378 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:38:43.571615 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] Deleted volume successfully. May 28 21:38:43.628483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-2654902a-9c2b-4a49-b6e0-d3ff45787cc9 tempest-AttachVolumeNegativeTest-679324601 None] Lock "b'cinder-9ae207c8-8c87-4201-942c-781ab5b960e5-delete_volume'" released by "delete_volume" :: held 3.651s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:38:46.979576 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-9859ae91-d651-4035-a8f2-0f853d20099c tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['58581991-8820-49dd-9ad5-d5a57f620bae', '9c05e271-03fb-4149-b603-0a0cd492a1cb', '7aa44418-c7a3-44cf-bbcc-60534d8e57bc', '090df421-ecc7-4119-b9fd-98338984804b'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:38:47.188725 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-9859ae91-d651-4035-a8f2-0f853d20099c tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:38:47.196551 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-9859ae91-d651-4035-a8f2-0f853d20099c tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-eb6b2fe0-bcf8-421c-b29a-2fb697798146-delete_snapshot'" released by "delete_snapshot" :: held 31.075s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:38:48.343652 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-49eb8ba8-013a-4123-b333-d30d16c3a6ac req-5f0fd29e-b3a2-4316-83c8-e6d97b1f44c4 tempest-VolumesSnapshotTestJSON-961770574 None] Terminate volume connection completed successfully. May 28 21:38:48.345677 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-49eb8ba8-013a-4123-b333-d30d16c3a6ac req-5f0fd29e-b3a2-4316-83c8-e6d97b1f44c4 tempest-VolumesSnapshotTestJSON-961770574 None] Deleting attachment ae758bd8-cdac-42a8-a96a-a4d23c3cbcda. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:38:53.144719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:38:53.166307 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (671354a2-5192-4187-b393-986c955fd259) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:38:53.172971 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bedc9d6a-f9ef-444d-968c-eb331133e1ee) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:53.196842 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (bedc9d6a-f9ef-444d-968c-eb331133e1ee) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:38:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-245735327',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=15b7d5e6-1381-490c-8627-480333392e46,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5833e78f486c4a118608290040b31cfc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:38:53Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:38:53Z,use_quota=True,user_id='388458a7b3504bd08f3789f53cf9ea35',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:53.203183 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (39ab4feb-1415-4492-a165-388cd13dcc50) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:53.207397 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (39ab4feb-1415-4492-a165-388cd13dcc50) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:53.212632 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20c1ba1b-b30c-4593-845f-3c6001d5d66d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:53.301397 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20c1ba1b-b30c-4593-845f-3c6001d5d66d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '15b7d5e6-1381-490c-8627-480333392e46', 'volume_name': 'volume-15b7d5e6-1381-490c-8627-480333392e46', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:53.306323 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a9bc9533-d5b3-4b8a-96f0-6187ea0d6720) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:53.310737 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a9bc9533-d5b3-4b8a-96f0-6187ea0d6720) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:53.315475 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9464b588-2bc8-4505-ac71-7f5243f89e70) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:53.316619 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Volume 15b7d5e6-1381-490c-8627-480333392e46: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-15b7d5e6-1381-490c-8627-480333392e46', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:38:53.317188 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Cloning 15b7d5e6-1381-490c-8627-480333392e46 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:38:53.326896 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:38:53.329305 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=55,cinder:UPDATE=38,cinder:INSERT=6,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:38:53.453553 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 38, 53, 331653)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:38:53.454017 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:38:53.454412 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:38:53.459576 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.133s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:38:53.460624 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:38:53.466622 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 38, 53, 464150)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:38:53.467030 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:38:53.467410 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:38:53.479328 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:53.738585 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] creating snapshot='volume-15b7d5e6-1381-490c-8627-480333392e46.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:38:55.579151 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-15b7d5e6-1381-490c-8627-480333392e46.clone_snap' to 'volume-15b7d5e6-1381-490c-8627-480333392e46' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:38:55.720310 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:38:55.746479 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (601a0035-6dc1-4ca3-a5af-8c99b5cdfbbc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:38:55.752732 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (11b5894e-06cc-4820-a7bd-3b82f052e54a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:55.770702 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (11b5894e-06cc-4820-a7bd-3b82f052e54a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:38:53Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-275646850',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=f5067dd4-140f-4988-9f8a-940125773cfb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6ed82f578e214948a9311cbdbb55a027',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:38:54Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:38:54Z,use_quota=True,user_id='8d033e99f9294e348b01c59a970931d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:55.779241 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cfb7c3ac-0f52-4e46-81de-d59d0f01a762) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:55.784097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cfb7c3ac-0f52-4e46-81de-d59d0f01a762) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:55.791307 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (753f16a0-d162-48fd-b724-4cdacf254b56) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:55.796578 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (753f16a0-d162-48fd-b724-4cdacf254b56) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f5067dd4-140f-4988-9f8a-940125773cfb', 'volume_name': 'volume-f5067dd4-140f-4988-9f8a-940125773cfb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:55.802133 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80d3554e-4ff7-442d-9c46-d363737e1352) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:55.807798 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (80d3554e-4ff7-442d-9c46-d363737e1352) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:55.813002 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (80e5d95f-1541-4722-8814-9ae25f770c9f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:55.814225 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Volume f5067dd4-140f-4988-9f8a-940125773cfb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f5067dd4-140f-4988-9f8a-940125773cfb', 'volume_size': 1} May 28 21:38:55.814703 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] creating volume 'volume-f5067dd4-140f-4988-9f8a-940125773cfb' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:38:55.815396 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:56.593702 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:38:56.595004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:38:56.599460 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (80e5d95f-1541-4722-8814-9ae25f770c9f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f5067dd4-140f-4988-9f8a-940125773cfb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:56.604658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (baa08e54-2053-450a-b88b-50b534257447) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:56.708857 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-f5067dd4-140f-4988-9f8a-940125773cfb (f5067dd4-140f-4988-9f8a-940125773cfb): created successfully May 28 21:38:56.708857 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (baa08e54-2053-450a-b88b-50b534257447) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:56.714200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (601a0035-6dc1-4ca3-a5af-8c99b5cdfbbc) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:38:56.797166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:38:56.798599 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:38:56.799083 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:38:57.083216 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:38:57.083844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Marking volume 15b7d5e6-1381-490c-8627-480333392e46 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:38:57.089188 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b0fc5aa7-8118-4871-a18d-2fbcf292b12e tempest-VolumesSnapshotTestJSON-961770574 None] Created volume successfully. May 28 21:38:57.159466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 15b7d5e6-1381-490c-8627-480333392e46. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:38:57.159906 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Creating volume glance metadata for volume 15b7d5e6-1381-490c-8627-480333392e46 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:38:57.265921 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (9464b588-2bc8-4505-ac71-7f5243f89e70) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-15b7d5e6-1381-490c-8627-480333392e46', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:57.270440 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (779ba505-29f6-4bfd-a63a-71ec162d65e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:38:57.343029 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-15b7d5e6-1381-490c-8627-480333392e46 (15b7d5e6-1381-490c-8627-480333392e46): created successfully May 28 21:38:57.346707 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (779ba505-29f6-4bfd-a63a-71ec162d65e6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:38:57.352184 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (671354a2-5192-4187-b393-986c955fd259) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:38:57.424126 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-241923b4-52bf-4a2b-beb2-bcfdc57beee7 tempest-TestVolumeBootPattern-1994669512 None] Created volume successfully. May 28 21:38:57.542745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f18d75d0-b654-43fd-967c-a2cac0a17b86 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:58.926891 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b6c1e641-4ee0-4f3c-a0c5-a62c71f3002d tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:38:59.955253 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f18d75d0-b654-43fd-967c-a2cac0a17b86 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:39:01.089217 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:39:01.090370 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:01.116115 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:01.142073 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:01.327026 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b6c1e641-4ee0-4f3c-a0c5-a62c71f3002d tempest-TestVolumeBootPattern-1994669512 None] Create snapshot completed successfully May 28 21:39:01.499958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:39:01.527815 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (88224060-aaa5-456a-a975-7dc34a8a9fe2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:01.534521 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (882f928d-080d-4959-8643-453cd583bc41) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:01.551167 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (882f928d-080d-4959-8643-453cd583bc41) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:39:01Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-368239837',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=66bb12a0-413f-4814-9d8d-d4891376d16a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6ed82f578e214948a9311cbdbb55a027',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:39:01Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=86d95156-0bf8-401c-a2be-89fa788104e2,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:39:01Z,use_quota=True,user_id='8d033e99f9294e348b01c59a970931d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:01.556499 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (302af109-0c7b-4f2d-b087-7b37903f05fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:01.560527 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (302af109-0c7b-4f2d-b087-7b37903f05fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:01.566273 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d1d0da6c-2c8a-4724-a45a-ae9d38ceff34) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:01.570174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d1d0da6c-2c8a-4724-a45a-ae9d38ceff34) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '66bb12a0-413f-4814-9d8d-d4891376d16a', 'volume_name': 'volume-66bb12a0-413f-4814-9d8d-d4891376d16a', 'volume_size': 2, 'snapshot_id': '86d95156-0bf8-401c-a2be-89fa788104e2'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:01.575067 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac0714cf-5d0c-43fc-8606-1446c1cabb4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:01.578903 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ac0714cf-5d0c-43fc-8606-1446c1cabb4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:01.583505 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f2499090-e454-4a3d-8ba0-7b3d65f2101a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:01.584572 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Volume 66bb12a0-413f-4814-9d8d-d4891376d16a: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-66bb12a0-413f-4814-9d8d-d4891376d16a', 'volume_size': 2, 'snapshot_id': '86d95156-0bf8-401c-a2be-89fa788104e2'} May 28 21:39:01.604564 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] cloning volumes/volume-f5067dd4-140f-4988-9f8a-940125773cfb@snapshot-86d95156-0bf8-401c-a2be-89fa788104e2 to volume-66bb12a0-413f-4814-9d8d-d4891376d16a {{(pid=135921) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1098}} May 28 21:39:01.605238 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:01.640426 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:02.447156 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:39:02.447728 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:39:02.453475 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:03.275110 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f2499090-e454-4a3d-8ba0-7b3d65f2101a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-66bb12a0-413f-4814-9d8d-d4891376d16a', 'volume_size': 2, 'snapshot_id': '86d95156-0bf8-401c-a2be-89fa788104e2'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:03.279736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f5bd3e0f-a0d9-4c7e-afc2-9254a194e829) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:03.399395 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-66bb12a0-413f-4814-9d8d-d4891376d16a (66bb12a0-413f-4814-9d8d-d4891376d16a): created successfully May 28 21:39:03.399395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f5bd3e0f-a0d9-4c7e-afc2-9254a194e829) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:03.404758 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (88224060-aaa5-456a-a975-7dc34a8a9fe2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:03.510081 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5be7647a-26c3-496c-bb28-bec249273019 tempest-VolumesSnapshotTestJSON-961770574 None] Created volume successfully. May 28 21:39:04.588968 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-66bb12a0-413f-4814-9d8d-d4891376d16a-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:04.723540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:04.932575 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:05.025087 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-66bb12a0-413f-4814-9d8d-d4891376d16a {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:06.698319 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=62,cinder:DELETE=5,cinder:UPDATE=18,cinder:INSERT=15 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:06.844580 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:39:06.874398 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (5f8b389f-b56c-466f-b4f8-f9af24487012) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:06.881446 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba2c57f6-d731-4425-88ba-cc2b2a6c92aa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:06.902006 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba2c57f6-d731-4425-88ba-cc2b2a6c92aa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:39:06Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=cc18c64f-cd5d-4009-abfb-00e8763d2d48,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5833e78f486c4a118608290040b31cfc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:39:07Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=5c75e3f3-953a-49e7-95ef-9ce38c375765,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:39:07Z,use_quota=True,user_id='388458a7b3504bd08f3789f53cf9ea35',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:06.908661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (273dd32c-f20c-4cc6-a0eb-61dfe248ffd0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:06.913704 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (273dd32c-f20c-4cc6-a0eb-61dfe248ffd0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:06.919260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5342cf1d-3abb-481e-bb54-589a2e5d7a93) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:06.924961 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5342cf1d-3abb-481e-bb54-589a2e5d7a93) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'volume_name': 'volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'volume_size': 1, 'snapshot_id': '5c75e3f3-953a-49e7-95ef-9ce38c375765'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:06.930454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c159eadd-92a7-4642-b399-3ca688794506) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:06.935246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c159eadd-92a7-4642-b399-3ca688794506) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:06.940224 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (218fd21b-884b-40a8-b671-6204ca630787) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:06.941647 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Volume cc18c64f-cd5d-4009-abfb-00e8763d2d48: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'volume_size': 1, 'snapshot_id': '5c75e3f3-953a-49e7-95ef-9ce38c375765'} May 28 21:39:06.970403 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] cloning volumes/volume-15b7d5e6-1381-490c-8627-480333392e46@snapshot-5c75e3f3-953a-49e7-95ef-9ce38c375765 to volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48 {{(pid=135921) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1098}} May 28 21:39:06.971443 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:07.117267 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:08.481553 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:39:08.482271 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:39:08.498049 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Marking volume cc18c64f-cd5d-4009-abfb-00e8763d2d48 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:39:08.551222 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Copying metadata from snapshot 5c75e3f3-953a-49e7-95ef-9ce38c375765 to cc18c64f-cd5d-4009-abfb-00e8763d2d48. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:429}} May 28 21:39:08.630119 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['9eb09769-4893-4a8b-bcab-b86f8ffc5b21', 'edfb450f-ae6b-44e0-b519-b399a6d7d5cc', '9129d737-62b2-4b4e-bfe2-c642b046f58d', '94e389e1-971e-4e99-a650-eb47c9451236'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:08.673059 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (218fd21b-884b-40a8-b671-6204ca630787) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'volume_size': 1, 'snapshot_id': '5c75e3f3-953a-49e7-95ef-9ce38c375765'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:08.677882 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (403797f1-8c13-414e-84e0-214e20372f91) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:08.744252 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48 (cc18c64f-cd5d-4009-abfb-00e8763d2d48): created successfully May 28 21:39:08.751123 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (403797f1-8c13-414e-84e0-214e20372f91) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:08.754142 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (5f8b389f-b56c-466f-b4f8-f9af24487012) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:08.784275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:08.823974 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:08.853890 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:08.859461 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:39:08.865751 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-13ff9ad0-a2be-4181-96e0-f9db2efe8b82 tempest-TestVolumeBootPattern-1994669512 None] Created volume successfully. May 28 21:39:08.954773 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-483f37e2-d666-42e0-964e-a9f655ee740b tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-66bb12a0-413f-4814-9d8d-d4891376d16a-delete_volume'" released by "delete_volume" :: held 4.366s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:09.572070 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=65,cinder:UPDATE=14,cinder:INSERT=18,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:09.626295 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:39:09.644756 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (cdbd8c9c-0449-4985-a45e-fc95da0768fe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:09.651395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (46916030-1a11-49cd-abc6-a2032df22f0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:09.667469 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (46916030-1a11-49cd-abc6-a2032df22f0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:39:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-699569255',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=28771566-9461-431b-b157-c127a3a06428,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6ed82f578e214948a9311cbdbb55a027',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:39:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:39:09Z,use_quota=True,user_id='8d033e99f9294e348b01c59a970931d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:09.673314 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b3e4b40c-6641-4611-8e43-b6c51406d231) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:09.677406 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b3e4b40c-6641-4611-8e43-b6c51406d231) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:09.682536 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3a9a53cb-d799-4dc8-9245-d73c7a66a2b8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:09.686898 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3a9a53cb-d799-4dc8-9245-d73c7a66a2b8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '28771566-9461-431b-b157-c127a3a06428', 'volume_name': 'volume-28771566-9461-431b-b157-c127a3a06428', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:09.691295 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e0fd361e-815d-4e14-8061-f58dd56ea63a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:09.695189 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e0fd361e-815d-4e14-8061-f58dd56ea63a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:09.699856 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b34f8246-f7d9-4b8c-a508-1d00f2e453fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:09.705564 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Volume 28771566-9461-431b-b157-c127a3a06428: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-28771566-9461-431b-b157-c127a3a06428', 'volume_size': 1} May 28 21:39:09.705564 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] creating volume 'volume-28771566-9461-431b-b157-c127a3a06428' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:39:09.705564 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:10.000458 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-2d46df74-8075-4723-872d-b53957349f37 req-95bbaa84-aba1-40be-8660-ecffe40e5ba4 tempest-TestVolumeBootPattern-1994669512 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:39:10.162304 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:39:10.162794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:39:10.167553 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b34f8246-f7d9-4b8c-a508-1d00f2e453fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-28771566-9461-431b-b157-c127a3a06428', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:10.173103 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3efc0572-c452-4215-bc1a-f285271ce339) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:10.256924 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-28771566-9461-431b-b157-c127a3a06428 (28771566-9461-431b-b157-c127a3a06428): created successfully May 28 21:39:10.260653 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (3efc0572-c452-4215-bc1a-f285271ce339) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:10.266962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (cdbd8c9c-0449-4985-a45e-fc95da0768fe) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:10.336786 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-6c5ab252-0be5-4b48-a51d-92684b9ea3ba tempest-VolumesSnapshotTestJSON-961770574 None] Created volume successfully. May 28 21:39:10.691377 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-2d46df74-8075-4723-872d-b53957349f37 req-95bbaa84-aba1-40be-8660-ecffe40e5ba4 tempest-TestVolumeBootPattern-1994669512 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.691s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:39:10.692061 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d46df74-8075-4723-872d-b53957349f37 req-95bbaa84-aba1-40be-8660-ecffe40e5ba4 tempest-TestVolumeBootPattern-1994669512 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:39:10.776936 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-95bbaa84-aba1-40be-8660-ecffe40e5ba4 tempest-TestVolumeBootPattern-1994669512 None] Connection info returned from driver {'name': 'volumes/volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'cc18c64f-cd5d-4009-abfb-00e8763d2d48', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '37e61717-dbbd-422f-a342-528de7bbd256'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:39:10.910885 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2d46df74-8075-4723-872d-b53957349f37 req-95bbaa84-aba1-40be-8660-ecffe40e5ba4 tempest-TestVolumeBootPattern-1994669512 None] attachment_update completed successfully. May 28 21:39:11.393369 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-634a39ff-3f05-4891-b96e-1c04823b11b7 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:13.995202 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-634a39ff-3f05-4891-b96e-1c04823b11b7 tempest-VolumesSnapshotTestJSON-961770574 None] Create snapshot completed successfully May 28 21:39:15.012900 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:39:15.040097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (62c765ed-4223-4578-a8cc-f3e52f1f97c5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:15.046958 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (482457b2-b2b5-4f4f-b38b-955b229d3d44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:15.063835 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (482457b2-b2b5-4f4f-b38b-955b229d3d44) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:39:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1515437154',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='6ed82f578e214948a9311cbdbb55a027',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:39:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=59f39d00-cdc8-466c-a3af-cf7b70dd2985,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:39:15Z,use_quota=True,user_id='8d033e99f9294e348b01c59a970931d5',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:15.069687 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (578eeb3f-b2a4-423f-b6ef-f948a38e5294) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:15.073907 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (578eeb3f-b2a4-423f-b6ef-f948a38e5294) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:15.079567 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ad38c674-29ab-4d9d-b7a0-0d97a0360605) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:15.083608 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ad38c674-29ab-4d9d-b7a0-0d97a0360605) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187', 'volume_name': 'volume-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187', 'volume_size': 1, 'snapshot_id': '59f39d00-cdc8-466c-a3af-cf7b70dd2985'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:15.088527 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1215488c-5e6e-4489-b87b-0f18b08cc23e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:15.092746 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1215488c-5e6e-4489-b87b-0f18b08cc23e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:15.097678 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (66da1b82-1d10-4037-ac0a-69a0884207a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:15.099960 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Volume 37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187', 'volume_size': 1, 'snapshot_id': '59f39d00-cdc8-466c-a3af-cf7b70dd2985'} May 28 21:39:15.120919 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] cloning volumes/volume-28771566-9461-431b-b157-c127a3a06428@snapshot-59f39d00-cdc8-466c-a3af-cf7b70dd2985 to volume-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187 {{(pid=135921) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1098}} May 28 21:39:15.121584 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:15.160970 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:16.173794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:39:16.176094 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:39:16.193794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (66da1b82-1d10-4037-ac0a-69a0884207a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187', 'volume_size': 1, 'snapshot_id': '59f39d00-cdc8-466c-a3af-cf7b70dd2985'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:16.198504 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (19177242-668a-47ef-bfa2-92a119a28ddd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:16.281060 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187 (37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187): created successfully May 28 21:39:16.285006 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (19177242-668a-47ef-bfa2-92a119a28ddd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:16.290845 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Flow 'volume_create_manager' (62c765ed-4223-4578-a8cc-f3e52f1f97c5) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:16.383741 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ef4b2223-271f-48d9-a8a4-34f58081a7b0 tempest-VolumesSnapshotTestJSON-961770574 None] Created volume successfully. May 28 21:39:17.166786 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:17.266794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:17.395450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:17.442423 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:20.416810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['9237b583-6136-4924-b46c-23cc3510b872', 'd9471621-e8ca-47cd-80e3-9e7242fe7ce7', '52632ecb-6321-4e9f-9ce9-e945e7ab8942', '2b7cac5c-7789-4ee4-bc48-80560c9f6030'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:20.584246 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:20.612319 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:20.635184 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:20.639289 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:39:20.737021 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-d60d81b9-ed9c-4d8a-ba00-15dd25aeb27d tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-37c7c7f3-2b4f-4e3c-a6b2-5cc773c9d187-delete_volume'" released by "delete_volume" :: held 3.570s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:21.534554 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-03c5defe-ee9f-475b-b3f7-b3cb7e550021 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-59f39d00-cdc8-466c-a3af-cf7b70dd2985-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:21.536073 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-03c5defe-ee9f-475b-b3f7-b3cb7e550021 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:23.738295 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=60,cinder:UPDATE=27,cinder:INSERT=12,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:23.835561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-03c5defe-ee9f-475b-b3f7-b3cb7e550021 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['28862fbd-646a-40d1-a2b6-b552513a42a0', '45404062-2065-4b16-9043-2c55633670ca', 'ec0dc657-6051-4432-b8c3-f9973f73ceaa', '66d029eb-5b9a-4753-9597-042a97be3487'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:24.001123 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-03c5defe-ee9f-475b-b3f7-b3cb7e550021 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:39:24.007568 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-03c5defe-ee9f-475b-b3f7-b3cb7e550021 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-59f39d00-cdc8-466c-a3af-cf7b70dd2985-delete_snapshot'" released by "delete_snapshot" :: held 2.473s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:24.727082 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-ff73b93b-35a6-4b8b-8e89-6bc3e962ec99 tempest-TestVolumeBootPattern-1994669512 None] Terminate volume connection completed successfully. May 28 21:39:24.727502 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-ff73b93b-35a6-4b8b-8e89-6bc3e962ec99 tempest-TestVolumeBootPattern-1994669512 None] Deleting attachment 37e61717-dbbd-422f-a342-528de7bbd256. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:39:24.884470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-28771566-9461-431b-b157-c127a3a06428-delete_volume'" acquired by "delete_volume" :: waited 0.013s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:24.980214 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:25.143863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:25.144480 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-28771566-9461-431b-b157-c127a3a06428 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:39:25.201108 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-28771566-9461-431b-b157-c127a3a06428 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:25.215722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-cc18c64f-cd5d-4009-abfb-00e8763d2d48-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:25.337450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:25.549029 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:25.784969 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] deleting rbd volume volume-cc18c64f-cd5d-4009-abfb-00e8763d2d48 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:30.337300 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=67,cinder:DELETE=3,cinder:UPDATE=24,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:30.412300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['036c4efd-7076-457f-be8a-b553542d66ef', '397adcac-0cc8-47cc-845a-7f4d851bb778', '57f3f7a8-18cf-4808-8e78-05a618633229', '1ace0991-2c9a-4d5b-9dfa-93aded4b15c3'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:30.584069 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:30.607215 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:30.631345 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:30.635859 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:39:30.708080 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-56b8339f-b23d-4f60-bbb3-a3222a849188 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-28771566-9461-431b-b157-c127a3a06428-delete_volume'" released by "delete_volume" :: held 5.824s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:30.749044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['503a01b9-1ffc-47fe-8d9e-61b0f8a7913b', '3cf1c441-2461-48cb-9759-47c27d906bd0', '31c6af0f-7a8e-4e67-afba-fe88e974f54a', 'a6f905eb-8f1c-42e3-8ed3-c450fd0da1fe'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:30.881452 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:30.904831 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:31.000140 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:31.004408 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] Deleted volume successfully. May 28 21:39:31.101389 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-cce325c5-6b2d-4e6a-8f4f-914078a7dfab req-c65b6a8d-c9ee-4a5b-92bc-699b513187bf tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-cc18c64f-cd5d-4009-abfb-00e8763d2d48-delete_volume'" released by "delete_volume" :: held 5.886s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:31.385532 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-76729514-ac65-4e51-ba1e-81777a231f7e tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-86d95156-0bf8-401c-a2be-89fa788104e2-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:31.387166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-76729514-ac65-4e51-ba1e-81777a231f7e tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:31.561810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-fc7e72ee-ae9a-4f2d-b6e3-b6e01bf72f94 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-5c75e3f3-953a-49e7-95ef-9ce38c375765-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:31.563237 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fc7e72ee-ae9a-4f2d-b6e3-b6e01bf72f94 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:34.538955 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-76729514-ac65-4e51-ba1e-81777a231f7e tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['d624cd2e-a8fa-41e9-b61e-42c786fa7a98', '54fedb9c-6107-444b-b57a-478400048c29', 'f6e8d3b8-a460-4faa-bfa0-bb66d98d19c5', 'd7b3475f-127b-44f6-b372-e0789ca43b53'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:34.541823 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-fc7e72ee-ae9a-4f2d-b6e3-b6e01bf72f94 tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['d5b0d07e-390a-4554-9b36-d44897c88a5f', '4fac011b-ada5-4a1e-86ac-2e8d9c5c4525', '2a9e4142-c7fd-4530-aff7-aa382f59ddb9', 'dc39b7c7-9e58-49d4-8308-3383440b6a38'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:34.763749 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-76729514-ac65-4e51-ba1e-81777a231f7e tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:39:34.769539 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=55,cinder:UPDATE=32,cinder:INSERT=12,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:34.772318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-76729514-ac65-4e51-ba1e-81777a231f7e tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-86d95156-0bf8-401c-a2be-89fa788104e2-delete_snapshot'" released by "delete_snapshot" :: held 3.386s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:34.835717 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fc7e72ee-ae9a-4f2d-b6e3-b6e01bf72f94 tempest-TestVolumeBootPattern-1994669512 None] Delete snapshot completed successfully. May 28 21:39:34.840294 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-fc7e72ee-ae9a-4f2d-b6e3-b6e01bf72f94 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-5c75e3f3-953a-49e7-95ef-9ce38c375765-delete_snapshot'" released by "delete_snapshot" :: held 3.279s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:35.704254 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-f5067dd4-140f-4988-9f8a-940125773cfb-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:35.821399 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:35.948950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:35.949422 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-f5067dd4-140f-4988-9f8a-940125773cfb is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:39:36.039421 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-f5067dd4-140f-4988-9f8a-940125773cfb {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:36.053745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-15b7d5e6-1381-490c-8627-480333392e46-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:36.148827 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:36.366371 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:36.510810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] deleting rbd volume volume-15b7d5e6-1381-490c-8627-480333392e46 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:40.938260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['18d4fc7c-7343-4bbf-a7b1-af82ae4e4abb', '9eced404-913b-4573-b43e-835b63a9f29f', '7c94a40d-0bc0-4ac8-b5fc-547345d06e51', '8a2aa167-6f8a-4a76-86ac-e5d764c2c2d5'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:40.967513 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:39:41.070950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] deleting parent snapshot volume-15b7d5e6-1381-490c-8627-480333392e46.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:39:42.953377 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-a420983a-b71a-4ca0-935a-b83c5153223d-delete_volume'" acquired by "delete_volume" :: waited 0.026s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:43.075151 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:43.251609 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:43.368065 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-a420983a-b71a-4ca0-935a-b83c5153223d {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:43.373194 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:43.404557 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:43.428954 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:43.435180 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:39:43.513274 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['1d6bfb65-8bbb-4e27-bc74-16637f3d3735', '97626de2-96d3-4d25-8d51-df9cd0f08254', 'a1ae1ae8-1d78-4326-847e-7f17ec07a170', 'c50f6afd-fdb6-40ad-b763-964649354a01'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:43.527231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-14145b34-5c84-483c-80fe-155d996ac01c tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-f5067dd4-140f-4988-9f8a-940125773cfb-delete_volume'" released by "delete_volume" :: held 7.823s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:43.738241 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:43.765399 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:43.791761 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:43.800050 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] Deleted volume successfully. May 28 21:39:43.945712 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1aa32b4a-8da8-4a1e-9690-2488e087cb2f tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-15b7d5e6-1381-490c-8627-480333392e46-delete_volume'" released by "delete_volume" :: held 7.892s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:45.311317 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:39:45.330921 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (d7975c56-eba6-437b-a09e-2beaa55f7017) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:45.337269 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (80821e43-2bb8-4aa2-a0ef-a32cf859a38d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:45.355805 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (80821e43-2bb8-4aa2-a0ef-a32cf859a38d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:39:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-1266138771',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=2869ef6c-a254-4e8c-adc3-29d4f49c8803,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5833e78f486c4a118608290040b31cfc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:39:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:39:45Z,use_quota=True,user_id='388458a7b3504bd08f3789f53cf9ea35',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:45.361154 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c3b39c78-7fb0-4161-a4b1-ef34cb32114d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:45.365264 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c3b39c78-7fb0-4161-a4b1-ef34cb32114d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:45.370472 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ba30f4a1-a0a1-485d-9cdd-681767d66b05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:45.455482 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ba30f4a1-a0a1-485d-9cdd-681767d66b05) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'volume_name': 'volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:45.460446 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8c662466-5848-470d-906d-9a7818927db8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:45.465275 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8c662466-5848-470d-906d-9a7818927db8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:45.470024 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3650241e-8875-4d67-a2da-a640ffb38989) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:45.476496 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Volume 2869ef6c-a254-4e8c-adc3-29d4f49c8803: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:39:45.476496 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Cloning 2869ef6c-a254-4e8c-adc3-29d4f49c8803 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:39:45.482316 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:45.559255 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 39, 45, 486981)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:39:45.559695 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:39:45.560099 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:39:45.564568 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.082s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:45.565495 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:39:45.658183 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 39, 45, 568981)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:39:45.658604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:39:45.658996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:39:45.700799 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:45.829329 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] creating snapshot='volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:39:47.664980 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803.clone_snap' to 'volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:39:47.669377 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:39:47.835051 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] deleting parent snapshot volume-a420983a-b71a-4ca0-935a-b83c5153223d.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:39:50.365364 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:39:50.367372 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:39:50.367809 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:39:50.657164 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:39:50.657794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Marking volume 2869ef6c-a254-4e8c-adc3-29d4f49c8803 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:39:50.663707 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=49,cinder:UPDATE=42,cinder:DELETE=2,cinder:INSERT=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:50.738231 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 2869ef6c-a254-4e8c-adc3-29d4f49c8803. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:39:50.738671 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Creating volume glance metadata for volume 2869ef6c-a254-4e8c-adc3-29d4f49c8803 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:39:50.758117 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['c7ea5c15-3ff8-4887-9199-40c51f2f9e0f', '87642194-d221-4ca7-85e2-37b14464823e', 'b1b0da46-9dd7-4858-bc47-b32514654de3', '4e43a2b6-d355-4572-aec6-c2ffc270461e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:50.763045 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=66,cinder:UPDATE=20,cinder:INSERT=13,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:39:50.929547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3650241e-8875-4d67-a2da-a640ffb38989) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:50.935046 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5e0fc09c-e945-4841-a37a-0a3b77a0ff79) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:39:50.956015 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:50.981950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:51.007515 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:51.012596 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:39:51.039815 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803 (2869ef6c-a254-4e8c-adc3-29d4f49c8803): created successfully May 28 21:39:51.044070 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5e0fc09c-e945-4841-a37a-0a3b77a0ff79) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:39:51.049498 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (d7975c56-eba6-437b-a09e-2beaa55f7017) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:39:51.086739 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-87ddb61d-d81b-4d31-8703-fd3dc9843241 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-c08d393b-3af5-4bf1-a0d6-9899b9419fa0-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:51.088997 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-87ddb61d-d81b-4d31-8703-fd3dc9843241 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:51.221273 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-7bdfaef5-f492-4e92-ad51-a9e2eecf7938 tempest-TestVolumeBootPattern-1994669512 None] Created volume successfully. May 28 21:39:51.227890 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-e9299280-0def-4fb0-ad83-cf8dc8a834aa tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-a420983a-b71a-4ca0-935a-b83c5153223d-delete_volume'" released by "delete_volume" :: held 8.275s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:53.084825 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-87ddb61d-d81b-4d31-8703-fd3dc9843241 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['8a98e10d-68d4-4c7b-927e-2a92d497bc1d', '1d62e655-0c27-402a-9363-24319abedfcc', '8b6a05b0-cc05-4214-8cf4-ab5e260fdedf', 'ae2abb30-f2f4-460e-b895-4936e2e26495'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:53.296802 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-87ddb61d-d81b-4d31-8703-fd3dc9843241 tempest-VolumesSnapshotTestJSON-961770574 None] Delete snapshot completed successfully. May 28 21:39:53.310010 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-87ddb61d-d81b-4d31-8703-fd3dc9843241 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-c08d393b-3af5-4bf1-a0d6-9899b9419fa0-delete_snapshot'" released by "delete_snapshot" :: held 2.216s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:39:54.412368 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-b31650fd-bab6-499a-bca7-af6b551f165e-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:39:54.576485 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:54.747264 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:39:54.747612 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] Volume volume-b31650fd-bab6-499a-bca7-af6b551f165e is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:39:54.881188 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] deleting rbd volume volume-b31650fd-bab6-499a-bca7-af6b551f165e {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:39:55.569124 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-e2809d68-68d2-499b-837e-186fd53f90ab req-d7483d1c-823f-40ce-bbf7-ac569875b494 tempest-TestVolumeBootPattern-1994669512 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:39:56.235197 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-e2809d68-68d2-499b-837e-186fd53f90ab req-d7483d1c-823f-40ce-bbf7-ac569875b494 tempest-TestVolumeBootPattern-1994669512 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.666s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:39:56.236030 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-e2809d68-68d2-499b-837e-186fd53f90ab req-d7483d1c-823f-40ce-bbf7-ac569875b494 tempest-TestVolumeBootPattern-1994669512 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:39:56.384366 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-e2809d68-68d2-499b-837e-186fd53f90ab req-d7483d1c-823f-40ce-bbf7-ac569875b494 tempest-TestVolumeBootPattern-1994669512 None] Connection info returned from driver {'name': 'volumes/volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '2869ef6c-a254-4e8c-adc3-29d4f49c8803', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'df550d90-1285-4cd8-9496-3b608ff40598'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:39:56.573318 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-e2809d68-68d2-499b-837e-186fd53f90ab req-d7483d1c-823f-40ce-bbf7-ac569875b494 tempest-TestVolumeBootPattern-1994669512 None] attachment_update completed successfully. May 28 21:39:57.952680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] Created reservations ['d1a0bfd2-c023-405b-b218-37cec787bff6', '6897952e-2a1f-4411-af07-ffc9923399ac', '4e7f4f3e-9cd4-4609-98c2-876a8e023111', 'f8ce035b-7308-4c91-a50a-cb59255f6aa3'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:39:58.224306 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:58.272928 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:39:58.301865 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:39:58.306294 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] Deleted volume successfully. May 28 21:39:58.387623 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-48cba03c-339a-4a3e-a3da-d85b490483e2 tempest-VolumesSnapshotTestJSON-961770574 None] Lock "b'cinder-b31650fd-bab6-499a-bca7-af6b551f165e-delete_volume'" released by "delete_volume" :: held 3.974s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:40:01.093305 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:40:01.094621 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:01.120869 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:01.143874 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:40:02.969633 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=27,cinder:INSERT=13,cinder:SELECT=57,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:03.053977 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:40:03.084398 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Flow 'volume_create_manager' (a288aa4e-6da6-4c60-865d-a0d37ea20e4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:40:03.091971 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3181093e-9579-4b41-8fbb-b9e400076a04) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:03.109680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3181093e-9579-4b41-8fbb-b9e400076a04) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:40:02Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotListTestJSON-Volume-1597526326',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ecb8b454-90cf-44ee-9401-a6c456d550ff,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8376a726a998426195c974ba76d614e8',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:40:03Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:40:03Z,use_quota=True,user_id='181b118771d64f09a894612ebcca32ac',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:03.115403 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f3e5b817-f290-4742-ab2d-587dedb7fee9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:03.120348 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f3e5b817-f290-4742-ab2d-587dedb7fee9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:03.125751 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (75b5d92e-3c47-4e68-a712-049cc819dc0b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:03.140267 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (75b5d92e-3c47-4e68-a712-049cc819dc0b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ecb8b454-90cf-44ee-9401-a6c456d550ff', 'volume_name': 'volume-ecb8b454-90cf-44ee-9401-a6c456d550ff', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:03.147828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (372df8e9-d51d-4b1b-88cc-71cbb398c59b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:03.152945 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (372df8e9-d51d-4b1b-88cc-71cbb398c59b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:03.158715 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c00ed25c-fc6a-4bc1-8ab7-00bda59b5442) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:03.159951 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Volume ecb8b454-90cf-44ee-9401-a6c456d550ff: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ecb8b454-90cf-44ee-9401-a6c456d550ff', 'volume_size': 1} May 28 21:40:03.160451 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] creating volume 'volume-ecb8b454-90cf-44ee-9401-a6c456d550ff' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:40:03.161445 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:03.666332 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:40:03.666703 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:40:03.670478 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c00ed25c-fc6a-4bc1-8ab7-00bda59b5442) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ecb8b454-90cf-44ee-9401-a6c456d550ff', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:03.681629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (eafa0fa6-37f1-44e8-a551-a887b2003fa4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:03.776764 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Volume volume-ecb8b454-90cf-44ee-9401-a6c456d550ff (ecb8b454-90cf-44ee-9401-a6c456d550ff): created successfully May 28 21:40:03.781770 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (eafa0fa6-37f1-44e8-a551-a887b2003fa4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:03.787691 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Flow 'volume_create_manager' (a288aa4e-6da6-4c60-865d-a0d37ea20e4a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:40:03.909375 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-f0d6103d-b249-4730-9315-c5e7b6e2eb47 tempest-VolumesSnapshotListTestJSON-1741504862 None] Created volume successfully. May 28 21:40:04.509091 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0e0482e9-03ff-4274-8a9f-cc934e4c7716 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:06.349178 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0e0482e9-03ff-4274-8a9f-cc934e4c7716 tempest-VolumesSnapshotListTestJSON-1741504862 None] Create snapshot completed successfully May 28 21:40:06.912354 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-4e42ccf9-8a9d-4ff0-91f5-f52cd629d6cb tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:08.158876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-00adc55c-1175-4596-80c7-7166ccd2b94e req-fdc56959-da67-418c-98a9-a51e5713664b tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:08.593940 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-4e42ccf9-8a9d-4ff0-91f5-f52cd629d6cb tempest-VolumesSnapshotListTestJSON-1741504862 None] Create snapshot completed successfully May 28 21:40:09.406158 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-49fdd217-7ff0-49d0-8afd-062ab2603b86 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:10.503377 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-00adc55c-1175-4596-80c7-7166ccd2b94e req-fdc56959-da67-418c-98a9-a51e5713664b tempest-TestVolumeBootPattern-1994669512 None] Create snapshot completed successfully May 28 21:40:11.679009 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-49fdd217-7ff0-49d0-8afd-062ab2603b86 tempest-VolumesSnapshotListTestJSON-1741504862 None] Create snapshot completed successfully May 28 21:40:13.175502 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5812590a-f698-4801-99ab-5c2bd4a1ca52 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-6c9237a6-2bd0-4d23-82e0-763d5948f270-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.015s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:40:13.177191 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-5812590a-f698-4801-99ab-5c2bd4a1ca52 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:15.574901 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:40:15.621536 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (a49d6c55-dcbf-4f15-8f03-8cb1ec8c06eb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:40:15.628165 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8fcd2cbb-f65a-4f0f-a65f-11fec2ebbdff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:15.648748 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8fcd2cbb-f65a-4f0f-a65f-11fec2ebbdff) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=True,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:40:15Z,deleted=False,deleted_at=None,display_description='',display_name='',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=e64c6abf-2ef6-4392-983f-c457762416fe,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5833e78f486c4a118608290040b31cfc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:40:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=2ca018f7-a0df-4767-af4d-8f3d3a002686,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:40:15Z,use_quota=True,user_id='388458a7b3504bd08f3789f53cf9ea35',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:15.654787 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (23f1438b-f7f7-4681-ba49-804a84824641) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:15.658793 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (23f1438b-f7f7-4681-ba49-804a84824641) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:15.663839 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a63c16b2-7e4f-485d-9ffc-ed4282c25764) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:15.671200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (a63c16b2-7e4f-485d-9ffc-ed4282c25764) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': 'e64c6abf-2ef6-4392-983f-c457762416fe', 'volume_name': 'volume-e64c6abf-2ef6-4392-983f-c457762416fe', 'volume_size': 1, 'snapshot_id': '2ca018f7-a0df-4767-af4d-8f3d3a002686'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:15.677083 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9c7b54b5-0c48-41be-8609-104c79449bd7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:15.681299 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (9c7b54b5-0c48-41be-8609-104c79449bd7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:15.686260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8ef3d21-059f-4e78-925d-8760aa2b3693) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:15.687500 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Volume e64c6abf-2ef6-4392-983f-c457762416fe: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-e64c6abf-2ef6-4392-983f-c457762416fe', 'volume_size': 1, 'snapshot_id': '2ca018f7-a0df-4767-af4d-8f3d3a002686'} May 28 21:40:15.700165 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=59,cinder:INSERT=9,cinder:UPDATE=28,cinder:DELETE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:15.731374 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] cloning volumes/volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803@snapshot-2ca018f7-a0df-4767-af4d-8f3d3a002686 to volume-e64c6abf-2ef6-4392-983f-c457762416fe {{(pid=135921) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1098}} May 28 21:40:15.732589 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:15.818036 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:15.954462 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-5812590a-f698-4801-99ab-5c2bd4a1ca52 tempest-VolumesSnapshotListTestJSON-1741504862 None] Created reservations ['2c27d973-9830-433e-afa5-4b4bb29c9f2f', 'ed607f24-472e-4392-b0e0-f5811c90d321', '98f2b5b9-3fc6-4b5d-9374-4540eb1fc28d', '194d82d8-44fc-4202-ba99-19b142304786'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:40:16.137794 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-5812590a-f698-4801-99ab-5c2bd4a1ca52 tempest-VolumesSnapshotListTestJSON-1741504862 None] Delete snapshot completed successfully. May 28 21:40:16.147596 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-5812590a-f698-4801-99ab-5c2bd4a1ca52 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-6c9237a6-2bd0-4d23-82e0-763d5948f270-delete_snapshot'" released by "delete_snapshot" :: held 2.968s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:40:16.495471 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-de18d08d-6bf9-46f7-b55f-cdc3c3af1ef6 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-85794b1b-40f2-4502-bbcc-2232f86ff4ee-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.031s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:40:16.495471 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-de18d08d-6bf9-46f7-b55f-cdc3c3af1ef6 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:17.192539 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:40:17.193975 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:40:17.219719 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Marking volume e64c6abf-2ef6-4392-983f-c457762416fe as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:40:17.286350 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Copying metadata from snapshot 2ca018f7-a0df-4767-af4d-8f3d3a002686 to e64c6abf-2ef6-4392-983f-c457762416fe. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:429}} May 28 21:40:17.443736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e8ef3d21-059f-4e78-925d-8760aa2b3693) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e64c6abf-2ef6-4392-983f-c457762416fe', 'volume_size': 1, 'snapshot_id': '2ca018f7-a0df-4767-af4d-8f3d3a002686'}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:17.449182 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac5a7182-e915-4c41-b116-4f82d7dc0ef1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:40:17.564622 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-e64c6abf-2ef6-4392-983f-c457762416fe (e64c6abf-2ef6-4392-983f-c457762416fe): created successfully May 28 21:40:17.568667 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ac5a7182-e915-4c41-b116-4f82d7dc0ef1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:40:17.575391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (a49d6c55-dcbf-4f15-8f03-8cb1ec8c06eb) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:40:17.675793 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-f72059c3-2012-42c6-b86b-4d2f2bf0a9b1 tempest-TestVolumeBootPattern-1994669512 None] Created volume successfully. May 28 21:40:18.913084 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-7904ee23-b509-4b97-83c7-c258abd5d751 tempest-TestVolumeBootPattern-1994669512 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:40:19.593309 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-7904ee23-b509-4b97-83c7-c258abd5d751 tempest-TestVolumeBootPattern-1994669512 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.680s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:40:19.594699 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-7904ee23-b509-4b97-83c7-c258abd5d751 tempest-TestVolumeBootPattern-1994669512 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e64c6abf-2ef6-4392-983f-c457762416fe', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'e64c6abf-2ef6-4392-983f-c457762416fe', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:40:19.720803 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-7904ee23-b509-4b97-83c7-c258abd5d751 tempest-TestVolumeBootPattern-1994669512 None] Connection info returned from driver {'name': 'volumes/volume-e64c6abf-2ef6-4392-983f-c457762416fe', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'e64c6abf-2ef6-4392-983f-c457762416fe', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '3ac0a70e-5ea1-4288-babd-202f91e68b01'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:40:19.788271 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=63,cinder:UPDATE=20,cinder:INSERT=16,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:19.903147 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-9657b22c-8514-42d8-8816-c33f6cdadb26 req-7904ee23-b509-4b97-83c7-c258abd5d751 tempest-TestVolumeBootPattern-1994669512 None] attachment_update completed successfully. May 28 21:40:19.918941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-de18d08d-6bf9-46f7-b55f-cdc3c3af1ef6 tempest-VolumesSnapshotListTestJSON-1741504862 None] Created reservations ['cc5ad51e-13e0-4c29-84d8-f540c78640cc', 'c1c6806c-0f82-4072-bdde-8fc591ee7f7b', '40efebeb-1dd6-481c-a25a-db504ec7fcde', '541f120c-87b9-4ca6-8193-a35a428b3594'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:40:20.109327 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-de18d08d-6bf9-46f7-b55f-cdc3c3af1ef6 tempest-VolumesSnapshotListTestJSON-1741504862 None] Delete snapshot completed successfully. May 28 21:40:20.115046 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-de18d08d-6bf9-46f7-b55f-cdc3c3af1ef6 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-85794b1b-40f2-4502-bbcc-2232f86ff4ee-delete_snapshot'" released by "delete_snapshot" :: held 3.623s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:40:20.786871 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-dcb60e6a-7256-4c95-8440-91787a8157c8 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-d1a755b2-375e-4609-9801-bc35f9f43358-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:40:20.789798 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dcb60e6a-7256-4c95-8440-91787a8157c8 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:22.358829 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-dcb60e6a-7256-4c95-8440-91787a8157c8 tempest-VolumesSnapshotListTestJSON-1741504862 None] Created reservations ['8114ceca-cb3e-4bfb-8a42-ab58edb7de1a', 'bb164b84-776b-40e7-a4d0-8d9f30cd865d', '6ba0e2c3-3767-4848-bf4f-23bdc4ac06e6', '9f946d18-beb4-43af-ba25-3070bbefefd2'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:40:22.542924 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dcb60e6a-7256-4c95-8440-91787a8157c8 tempest-VolumesSnapshotListTestJSON-1741504862 None] Delete snapshot completed successfully. May 28 21:40:22.552016 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-dcb60e6a-7256-4c95-8440-91787a8157c8 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-d1a755b2-375e-4609-9801-bc35f9f43358-delete_snapshot'" released by "delete_snapshot" :: held 1.765s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:40:23.118891 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-ecb8b454-90cf-44ee-9401-a6c456d550ff-delete_volume'" acquired by "delete_volume" :: waited 0.027s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:40:23.219912 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:23.362433 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:40:23.363056 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] Volume volume-ecb8b454-90cf-44ee-9401-a6c456d550ff is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:40:23.451981 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] deleting rbd volume volume-ecb8b454-90cf-44ee-9401-a6c456d550ff {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:40:25.919508 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=63,cinder:INSERT=14,cinder:UPDATE=22,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:26.013022 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] Created reservations ['e5e4b2a8-48f1-4eb4-b22e-f1b60fdfbcf6', '3051ad4d-60f9-4122-b60a-e0324ef73557', 'cacb86f7-de68-4acd-98f9-8010ac24c567', '8030399c-a929-4c71-ac30-fa58c9e7ab9a'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:40:26.225529 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:26.292122 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:26.359887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:40:26.379608 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] Deleted volume successfully. May 28 21:40:26.453973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-56aca354-3a92-47d9-94e6-520aeacbc1a4 tempest-VolumesSnapshotListTestJSON-1741504862 None] Lock "b'cinder-ecb8b454-90cf-44ee-9401-a6c456d550ff-delete_volume'" released by "delete_volume" :: held 3.335s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:40:34.164389 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-fd7188f2-c063-4a79-adca-44d29327d578 req-c4a3dfdb-bc46-4366-ad41-652d3f06629b tempest-TestVolumeBootPattern-1994669512 None] Terminate volume connection completed successfully. May 28 21:40:34.171075 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-fd7188f2-c063-4a79-adca-44d29327d578 req-c4a3dfdb-bc46-4366-ad41-652d3f06629b tempest-TestVolumeBootPattern-1994669512 None] Deleting attachment 3ac0a70e-5ea1-4288-babd-202f91e68b01. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:40:34.670318 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-e64c6abf-2ef6-4392-983f-c457762416fe-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:40:34.820280 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:35.130994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:40:35.316723 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] deleting rbd volume volume-e64c6abf-2ef6-4392-983f-c457762416fe {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:40:42.376455 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['73f07bc0-c138-4fac-967c-96b4cee690f0', 'f1b35c65-b3fa-49b0-89cd-97036ffa5f94', '5194b33e-942e-4d67-9b2f-5c28242b6e3f', '23a21125-c40c-4fbc-baa3-68314ce9d12f'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:40:42.636939 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:42.721424 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:42.918466 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:40:42.932116 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] Deleted volume successfully. May 28 21:40:43.053264 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-fd7188f2-c063-4a79-adca-44d29327d578 req-fb5dc392-788e-4fad-beea-550906c00d6f tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-e64c6abf-2ef6-4392-983f-c457762416fe-delete_volume'" released by "delete_volume" :: held 8.380s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:40:43.471645 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-aa395851-8959-4aec-ae92-b1b630cfde91 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-2ca018f7-a0df-4767-af4d-8f3d3a002686-delete_snapshot'" acquired by "delete_snapshot" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:40:43.473407 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-aa395851-8959-4aec-ae92-b1b630cfde91 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:40:57.690466 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=19,cinder:UPDATE=10,cinder:INSERT=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:40:57.692043 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=43,cinder:INSERT=4,cinder:UPDATE=28,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:01.089815 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:41:01.090527 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:41:01.125040 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:41:01.203417 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:41:09.308165 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:15.008098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-aa395851-8959-4aec-ae92-b1b630cfde91 tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['444da222-dc7a-41cf-b26c-2b2d31dbea5b', 'c2391eca-92a7-455d-8921-e8dbc59543f8', '1ee42f14-ed93-490a-9933-09cb93343769', 'a9d60a75-93ae-4325-afb9-b4c560b95245'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:41:15.375519 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-aa395851-8959-4aec-ae92-b1b630cfde91 tempest-TestVolumeBootPattern-1994669512 None] Delete snapshot completed successfully. May 28 21:41:15.381066 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-aa395851-8959-4aec-ae92-b1b630cfde91 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-2ca018f7-a0df-4767-af4d-8f3d3a002686-delete_snapshot'" released by "delete_snapshot" :: held 31.909s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:41:18.566738 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-c2cb7e21-02ad-4b73-ae67-9f84ea4e9524 tempest-TestVolumeBootPattern-1994669512 None] Terminate volume connection completed successfully. May 28 21:41:18.567894 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-c2cb7e21-02ad-4b73-ae67-9f84ea4e9524 tempest-TestVolumeBootPattern-1994669512 None] Deleting attachment df550d90-1285-4cd8-9496-3b608ff40598. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:41:18.995560 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-2869ef6c-a254-4e8c-adc3-29d4f49c8803-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:41:19.150118 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:41:19.366483 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:41:19.439211 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] deleting rbd volume volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:41:22.521926 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:41:22.641038 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] deleting parent snapshot volume-2869ef6c-a254-4e8c-adc3-29d4f49c8803.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:41:24.137287 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['db62ee1c-5207-4423-bc80-cf1032b61041', '0dbebe15-685d-4000-b707-482839641574', '43e0d72b-fb62-4010-b4ad-b49b10ccf5c0', '051d6474-bdc4-41ed-ad8e-9c0d25be55ec'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:41:24.341371 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:41:24.364986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:41:24.482098 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:41:24.489813 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] Deleted volume successfully. May 28 21:41:24.593584 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-f804261a-75b6-470f-a5cb-d5c31e102e2f req-31f6d826-6a86-411a-8375-986cbf3423df tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-2869ef6c-a254-4e8c-adc3-29d4f49c8803-delete_volume'" released by "delete_volume" :: held 5.598s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:41:30.438250 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=29,cinder:UPDATE=17,cinder:INSERT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:41:30.567124 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:41:30.587803 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (76bd1b95-e70f-43c3-bd95-c9d397839302) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:41:30.594461 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (281e882f-19ce-4a04-b16a-5094269b89de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:41:30.609867 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (281e882f-19ce-4a04-b16a-5094269b89de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:41:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestVolumeBootPattern-volume-origin-491967946',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=76158cb5-574c-466c-82f8-f0cc50204799,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='5833e78f486c4a118608290040b31cfc',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:41:30Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:41:30Z,use_quota=True,user_id='388458a7b3504bd08f3789f53cf9ea35',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:41:30.615082 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d3b1fa9b-00e9-4820-ac25-dad62cd5241c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:41:30.619085 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d3b1fa9b-00e9-4820-ac25-dad62cd5241c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:41:30.624104 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d2fd78bf-7b84-4bc5-821f-109f366fa1c8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:41:30.708359 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d2fd78bf-7b84-4bc5-821f-109f366fa1c8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '76158cb5-574c-466c-82f8-f0cc50204799', 'volume_name': 'volume-76158cb5-574c-466c-82f8-f0cc50204799', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:41:30.714675 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5307eaeb-327c-4320-af89-04ee326ad08e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:41:30.718709 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5307eaeb-327c-4320-af89-04ee326ad08e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:41:30.723442 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0583ff08-5a94-4a48-b6e7-9a879093e3f4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:41:30.724568 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Volume 76158cb5-574c-466c-82f8-f0cc50204799: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-76158cb5-574c-466c-82f8-f0cc50204799', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 21:41:30.725144 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Cloning 76158cb5-574c-466c-82f8-f0cc50204799 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 21:41:30.736005 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:41:30.816983 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 41, 30, 738654)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:41:30.817745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:41:30.818359 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 21:41:30.827102 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.090s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:41:30.827102 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 21:41:30.832065 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 21, 41, 30, 829801)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 21:41:30.832629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 21:41:30.833137 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 21:41:30.845676 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:41:30.959022 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] creating snapshot='volume-76158cb5-574c-466c-82f8-f0cc50204799.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 21:41:31.992260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-76158cb5-574c-466c-82f8-f0cc50204799.clone_snap' to 'volume-76158cb5-574c-466c-82f8-f0cc50204799' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 21:41:33.057794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:41:33.059597 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:41:33.060405 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:41:33.261068 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 21:41:33.261068 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Marking volume 76158cb5-574c-466c-82f8-f0cc50204799 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 21:41:33.363741 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to 76158cb5-574c-466c-82f8-f0cc50204799. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 21:41:33.364236 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Creating volume glance metadata for volume 76158cb5-574c-466c-82f8-f0cc50204799 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 21:41:33.517056 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0583ff08-5a94-4a48-b6e7-9a879093e3f4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-76158cb5-574c-466c-82f8-f0cc50204799', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:41:33.524051 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (38a14844-a460-4760-8909-4ed4a8f52032) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:41:33.621468 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Volume volume-76158cb5-574c-466c-82f8-f0cc50204799 (76158cb5-574c-466c-82f8-f0cc50204799): created successfully May 28 21:41:33.625250 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (38a14844-a460-4760-8909-4ed4a8f52032) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:41:33.631157 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Flow 'volume_create_manager' (76bd1b95-e70f-43c3-bd95-c9d397839302) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:41:33.758231 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ffcf8240-b582-44e8-8a86-c9e13ceb3d88 tempest-TestVolumeBootPattern-1994669512 None] Created volume successfully. May 28 21:41:38.756011 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-40ab84c1-b9d1-4def-bb61-9c5c7ec00988 req-bed8ddd5-b0bf-446d-8300-3d36549b9e6a tempest-TestVolumeBootPattern-1994669512 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:41:39.595000 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-40ab84c1-b9d1-4def-bb61-9c5c7ec00988 req-bed8ddd5-b0bf-446d-8300-3d36549b9e6a tempest-TestVolumeBootPattern-1994669512 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.838s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:41:39.596013 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-40ab84c1-b9d1-4def-bb61-9c5c7ec00988 req-bed8ddd5-b0bf-446d-8300-3d36549b9e6a tempest-TestVolumeBootPattern-1994669512 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-76158cb5-574c-466c-82f8-f0cc50204799', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '76158cb5-574c-466c-82f8-f0cc50204799', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:41:39.684429 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-40ab84c1-b9d1-4def-bb61-9c5c7ec00988 req-bed8ddd5-b0bf-446d-8300-3d36549b9e6a tempest-TestVolumeBootPattern-1994669512 None] Connection info returned from driver {'name': 'volumes/volume-76158cb5-574c-466c-82f8-f0cc50204799', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '76158cb5-574c-466c-82f8-f0cc50204799', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e1619758-598c-4403-822f-3be76a3403ca'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:41:39.826843 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-40ab84c1-b9d1-4def-bb61-9c5c7ec00988 req-bed8ddd5-b0bf-446d-8300-3d36549b9e6a tempest-TestVolumeBootPattern-1994669512 None] attachment_update completed successfully. May 28 21:41:47.704660 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=57,cinder:INSERT=11,cinder:UPDATE=20,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:01.090132 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:42:01.091073 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:42:01.140572 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:42:01.195174 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:42:07.703044 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=30,cinder:UPDATE=4,cinder:INSERT=6,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:17.709198 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:36.594328 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:42:36.614534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Flow 'volume_create_manager' (6cc33d00-3813-43ed-b431-792821fa4eff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:42:36.620963 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d200132b-83e1-4230-8478-485c331d772e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:36.636717 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d200132b-83e1-4230-8478-485c331d772e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:42:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1674854552',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=4bdda978-6c53-441e-936a-9adb0994478b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='201bd6cbfd144d5c9ff30c81341773c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:42:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:42:36Z,use_quota=True,user_id='a69e3e8e9a7c4eda80a8c110a62511e2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:36.642085 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (79c14bc1-286d-4604-b38f-dba1ea847b27) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:36.646210 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (79c14bc1-286d-4604-b38f-dba1ea847b27) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:36.651297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e393982-66f8-4f8d-b5fa-5205b779435e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:36.655814 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5e393982-66f8-4f8d-b5fa-5205b779435e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4bdda978-6c53-441e-936a-9adb0994478b', 'volume_name': 'volume-4bdda978-6c53-441e-936a-9adb0994478b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:36.660661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59420739-4592-47cb-893a-c4df44a09fea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:36.664733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (59420739-4592-47cb-893a-c4df44a09fea) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:36.670035 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (976e0bac-a74b-4afb-9171-377fe9476018) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:36.671095 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Volume 4bdda978-6c53-441e-936a-9adb0994478b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4bdda978-6c53-441e-936a-9adb0994478b', 'volume_size': 1} May 28 21:42:36.671511 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] creating volume 'volume-4bdda978-6c53-441e-936a-9adb0994478b' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:42:36.672341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:42:37.269589 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:42:37.269996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:42:37.273655 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (976e0bac-a74b-4afb-9171-377fe9476018) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4bdda978-6c53-441e-936a-9adb0994478b', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:37.278826 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (75c6ce8d-07ee-47fb-965d-10bb9bd98309) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:37.329648 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Volume volume-4bdda978-6c53-441e-936a-9adb0994478b (4bdda978-6c53-441e-936a-9adb0994478b): created successfully May 28 21:42:37.333286 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (75c6ce8d-07ee-47fb-965d-10bb9bd98309) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:37.338957 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Flow 'volume_create_manager' (6cc33d00-3813-43ed-b431-792821fa4eff) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:42:37.408096 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0dc270d2-498a-484c-ab06-d08037ea196f tempest-VolumesBackupsAdminTest-1665785387 None] Created volume successfully. May 28 21:42:38.240353 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:42:37Z,data_timestamp=2022-05-28T21:42:37Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-1613682014',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=37375bd5-f706-494e-8ceb-f6466da4fac9,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='201bd6cbfd144d5c9ff30c81341773c6',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='a69e3e8e9a7c4eda80a8c110a62511e2',volume_id=4bdda978-6c53-441e-936a-9adb0994478b) May 28 21:42:38.241193 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] continue_backup in rpcapi backup_id 37375bd5-f706-494e-8ceb-f6466da4fac9 {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:42:38.358045 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:42:38.983907 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.626s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:42:38.985239 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4bdda978-6c53-441e-936a-9adb0994478b', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '4bdda978-6c53-441e-936a-9adb0994478b', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:42:38.993910 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=11,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:39.001205 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] Initialize volume connection completed successfully. May 28 21:42:46.336784 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] Terminate volume connection completed successfully. May 28 21:42:46.354602 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-dbdac7db-da12-4977-8be1-76258d857134 tempest-VolumesBackupsAdminTest-1665785387 None] Remove volume export completed successfully. May 28 21:42:50.087357 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=16,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:42:50.163724 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:42:50.184080 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Flow 'volume_create_manager' (332fdb38-1b91-4cad-9a54-7e20aafa46ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:42:50.191135 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1da08e34-0d6e-4265-9aaf-66fd2e4bdcc4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:50.208665 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1da08e34-0d6e-4265-9aaf-66fd2e4bdcc4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:42:49Z,deleted=False,deleted_at=None,display_description='auto-created_from_restore_from_backup',display_name='restore_backup_37375bd5-f706-494e-8ceb-f6466da4fac9',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=1cd0b801-96f3-4e1c-b0ce-3742852cf0e7,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='201bd6cbfd144d5c9ff30c81341773c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:42:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:42:50Z,use_quota=True,user_id='a69e3e8e9a7c4eda80a8c110a62511e2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:50.214236 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (75a4f68e-9669-458e-9d67-712e03550a05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:50.218278 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (75a4f68e-9669-458e-9d67-712e03550a05) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:50.223292 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4520a006-d91c-4e49-80f1-184c3b0d9f22) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:50.227283 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4520a006-d91c-4e49-80f1-184c3b0d9f22) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1cd0b801-96f3-4e1c-b0ce-3742852cf0e7', 'volume_name': 'volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:50.232669 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d3573fbd-fb5d-426c-88ca-aa6a463530f1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:50.236694 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d3573fbd-fb5d-426c-88ca-aa6a463530f1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:50.241371 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fd94268f-d5b9-4b3a-98ce-598f7d8e81b2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:50.242474 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Volume 1cd0b801-96f3-4e1c-b0ce-3742852cf0e7: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7', 'volume_size': 1} May 28 21:42:50.242878 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] creating volume 'volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:42:50.243998 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:42:50.892539 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:42:50.892910 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:42:50.896507 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fd94268f-d5b9-4b3a-98ce-598f7d8e81b2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:50.901044 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2a3e5ff3-0556-45c9-9c50-47e4b1e4ade5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:42:50.979100 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Volume volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7 (1cd0b801-96f3-4e1c-b0ce-3742852cf0e7): created successfully May 28 21:42:50.983125 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2a3e5ff3-0556-45c9-9c50-47e4b1e4ade5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:42:50.988362 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Flow 'volume_create_manager' (332fdb38-1b91-4cad-9a54-7e20aafa46ba) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:42:51.078107 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Created volume successfully. May 28 21:42:52.170859 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:42:52.790771 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.620s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:42:52.791587 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '1cd0b801-96f3-4e1c-b0ce-3742852cf0e7', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:42:52.807722 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Initialize volume connection completed successfully. May 28 21:42:56.081970 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Terminate volume connection completed successfully. May 28 21:42:56.099282 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-60deed66-4cac-44bb-8cef-8765a60673a2 tempest-VolumesBackupsAdminTest-1665785387 None] Remove volume export completed successfully. May 28 21:42:57.573078 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] Lock "b'cinder-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:42:57.661336 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:42:59.681671 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] Volume volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:42:59.924388 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] deleting rbd volume volume-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:43:01.092173 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:43:01.092996 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:01.121029 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:01.144786 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:43:04.717830 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] Created reservations ['60332046-2d08-4e1f-81a2-e2e53b1612af', 'f96202af-efd6-446a-b729-d39aa33836d0', 'ba5ea25f-e535-4d89-a88a-1e2c3f6328c0', '9153a882-9855-499d-9e87-76eab874d022'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:43:04.918311 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:04.940279 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:04.963916 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:43:04.974029 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] Deleted volume successfully. May 28 21:43:05.053217 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-508269ac-5dff-4dc0-8f13-e82786a78c56 tempest-VolumesBackupsAdminTest-1665785387 None] Lock "b'cinder-1cd0b801-96f3-4e1c-b0ce-3742852cf0e7-delete_volume'" released by "delete_volume" :: held 7.480s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:43:07.902601 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:43:07.922500 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Flow 'volume_create_manager' (bf4655bd-411a-4c9a-bc37-d1de2262dba4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:43:07.928824 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d6739867-8dd0-4c1f-be94-76661878bc75) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:07.944506 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (d6739867-8dd0-4c1f-be94-76661878bc75) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:43:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Volume-1488234154',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=29650c1b-4d9b-4ef4-87e6-128966066aa2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='201bd6cbfd144d5c9ff30c81341773c6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:43:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:43:08Z,use_quota=True,user_id='a69e3e8e9a7c4eda80a8c110a62511e2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:07.949776 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (851feeac-a3c6-4e5c-9ebc-653d6649a428) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:07.954561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (851feeac-a3c6-4e5c-9ebc-653d6649a428) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:07.959614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1c3aab7b-a92e-46a2-b240-7a1d9afbcc80) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:07.963876 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (1c3aab7b-a92e-46a2-b240-7a1d9afbcc80) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '29650c1b-4d9b-4ef4-87e6-128966066aa2', 'volume_name': 'volume-29650c1b-4d9b-4ef4-87e6-128966066aa2', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:07.968827 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (39dea4f2-e2ee-4dfb-b0a6-819d75006a1d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:07.972722 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (39dea4f2-e2ee-4dfb-b0a6-819d75006a1d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:07.977364 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2aa068a-171c-44f6-ae0f-9d9f8410a7d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:07.978477 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Volume 29650c1b-4d9b-4ef4-87e6-128966066aa2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-29650c1b-4d9b-4ef4-87e6-128966066aa2', 'volume_size': 1} May 28 21:43:07.978865 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] creating volume 'volume-29650c1b-4d9b-4ef4-87e6-128966066aa2' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:43:07.979609 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:08.633735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:43:08.634076 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:43:08.637798 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (a2aa068a-171c-44f6-ae0f-9d9f8410a7d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-29650c1b-4d9b-4ef4-87e6-128966066aa2', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:08.642983 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (813e9e2a-658c-4e54-9633-b6ea3c877e30) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:08.724540 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Volume volume-29650c1b-4d9b-4ef4-87e6-128966066aa2 (29650c1b-4d9b-4ef4-87e6-128966066aa2): created successfully May 28 21:43:08.728777 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (813e9e2a-658c-4e54-9633-b6ea3c877e30) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:08.735811 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Flow 'volume_create_manager' (bf4655bd-411a-4c9a-bc37-d1de2262dba4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:43:08.820692 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fdf008af-6484-4c18-8187-4c0999e1d63a tempest-VolumesBackupsAdminTest-1665785387 None] Created volume successfully. May 28 21:43:09.937509 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=31,cinder:UPDATE=16,cinder:INSERT=3,cinder:DELETE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:10.338545 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-05-28T21:43:09Z,data_timestamp=2022-05-28T21:43:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsAdminTest-Backup-1710208003',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=91dc451b-cfbb-497c-915d-e958a807681b,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='201bd6cbfd144d5c9ff30c81341773c6',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='a69e3e8e9a7c4eda80a8c110a62511e2',volume_id=29650c1b-4d9b-4ef4-87e6-128966066aa2) May 28 21:43:10.339341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.backup.rpcapi [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] continue_backup in rpcapi backup_id 91dc451b-cfbb-497c-915d-e958a807681b {{(pid=135921) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}} May 28 21:43:10.454029 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:43:11.064210 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.610s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:43:11.065632 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-29650c1b-4d9b-4ef4-87e6-128966066aa2', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': '29650c1b-4d9b-4ef4-87e6-128966066aa2', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:43:11.080989 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] Initialize volume connection completed successfully. May 28 21:43:16.106484 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] Terminate volume connection completed successfully. May 28 21:43:16.122854 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-eb69de26-5ac3-44f4-92f8-3782776dcdc3 tempest-VolumesBackupsAdminTest-1665785387 None] Remove volume export completed successfully. May 28 21:43:27.069083 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] Lock "b'cinder-29650c1b-4d9b-4ef4-87e6-128966066aa2-delete_volume'" acquired by "delete_volume" :: waited 0.011s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:43:27.071067 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=38,cinder:DELETE=1,cinder:INSERT=1,cinder:UPDATE=8 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:27.195977 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:29.634268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] Volume volume-29650c1b-4d9b-4ef4-87e6-128966066aa2 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:43:30.122478 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] deleting rbd volume volume-29650c1b-4d9b-4ef4-87e6-128966066aa2 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:43:34.557819 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] Created reservations ['772977db-b996-4f3b-be01-37a57837d502', 'f0652e2f-eb04-4737-b316-e7ddc301baf3', '70de8fca-f0ab-415a-a919-e468a27647c1', 'b1764fb9-a0f6-4975-a4f2-c442ed23d748'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:43:34.712930 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:34.736321 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:34.759248 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:43:34.763563 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] Deleted volume successfully. May 28 21:43:34.932900 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-1f38013d-4c1d-4141-b438-9a2c7661eb61 tempest-VolumesBackupsAdminTest-1665785387 None] Lock "b'cinder-29650c1b-4d9b-4ef4-87e6-128966066aa2-delete_volume'" released by "delete_volume" :: held 7.865s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:43:35.693213 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] Lock "b'cinder-4bdda978-6c53-441e-936a-9adb0994478b-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:43:35.780834 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:38.063645 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] Volume volume-4bdda978-6c53-441e-936a-9adb0994478b is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:43:38.689610 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] deleting rbd volume volume-4bdda978-6c53-441e-936a-9adb0994478b {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:43:43.327697 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=35,cinder:UPDATE=10,cinder:INSERT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:43:43.412972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] Created reservations ['1a4576fd-bbaf-4e4b-9a48-f6cc4071d8c7', '402da01d-90c3-44f8-b085-ced57e923083', '17889816-6ff9-40d9-95e1-ed1dff89277b', 'a9c2a74b-3d02-4cef-8918-5a4668710d33'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:43:43.598760 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:43.629094 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:43.651844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:43:43.655990 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] Deleted volume successfully. May 28 21:43:43.744771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-a5695356-7ae3-48ee-8a0e-e68e5685e4c3 tempest-VolumesBackupsAdminTest-1665785387 None] Lock "b'cinder-4bdda978-6c53-441e-936a-9adb0994478b-delete_volume'" released by "delete_volume" :: held 8.052s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:43:49.877054 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:43:49.896221 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Flow 'volume_create_manager' (4207877b-d4ee-4272-b642-7cbf6f8a11dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:43:49.902950 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5f6c4da1-0f0b-445d-bdc7-10e5729c4850) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:49.918689 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (5f6c4da1-0f0b-445d-bdc7-10e5729c4850) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:43:49Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-297401892',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=62903144-b4f9-4936-8de2-f1d9f198c2bb,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='65965326d8ef4ea89a8a4245e147bee6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:43:50Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:43:50Z,use_quota=True,user_id='a436e059356f4840ac977fcc5db9d5b3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:49.923910 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (19e91572-e038-470e-a513-7cd0946e983d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:49.927968 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (19e91572-e038-470e-a513-7cd0946e983d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:49.933629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b0c84e6-a7c7-48bd-b05c-95537edac45f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:49.937574 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7b0c84e6-a7c7-48bd-b05c-95537edac45f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '62903144-b4f9-4936-8de2-f1d9f198c2bb', 'volume_name': 'volume-62903144-b4f9-4936-8de2-f1d9f198c2bb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:49.942385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e93ba2b8-62fa-4efb-bdbb-5b2c29bfd10d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:49.946242 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (e93ba2b8-62fa-4efb-bdbb-5b2c29bfd10d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:49.950973 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7ae96811-517c-4997-85c9-72c63486c52d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:49.952046 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Volume 62903144-b4f9-4936-8de2-f1d9f198c2bb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-62903144-b4f9-4936-8de2-f1d9f198c2bb', 'volume_size': 1} May 28 21:43:49.952436 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] creating volume 'volume-62903144-b4f9-4936-8de2-f1d9f198c2bb' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:43:49.953178 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:50.672652 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:43:50.673027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:43:50.676680 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7ae96811-517c-4997-85c9-72c63486c52d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-62903144-b4f9-4936-8de2-f1d9f198c2bb', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:50.681249 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c6973a6-c1d0-4a8e-9714-13ca00008368) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:43:50.778263 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Volume volume-62903144-b4f9-4936-8de2-f1d9f198c2bb (62903144-b4f9-4936-8de2-f1d9f198c2bb): created successfully May 28 21:43:50.781934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c6973a6-c1d0-4a8e-9714-13ca00008368) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:43:50.787446 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Flow 'volume_create_manager' (4207877b-d4ee-4272-b642-7cbf6f8a11dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:43:50.915661 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ac05d46b-7ea6-4ded-8d95-cbe6b8ad57d5 tempest-VolumesTransfersV355Test-1675622206 None] Created volume successfully. May 28 21:43:52.986025 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-7c36157d-5b36-4f2f-9544-9e3932ed4d3e tempest-VolumesTransfersV355Test-1612540294 None] Transfer volume completed successfully. May 28 21:43:53.975166 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] Lock "b'cinder-62903144-b4f9-4936-8de2-f1d9f198c2bb-delete_volume'" acquired by "delete_volume" :: waited 0.012s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:43:54.065863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:54.240661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:43:54.240993 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] Volume volume-62903144-b4f9-4936-8de2-f1d9f198c2bb is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:43:54.391534 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] deleting rbd volume volume-62903144-b4f9-4936-8de2-f1d9f198c2bb {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:43:59.046726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] Created reservations ['55b0066c-81f3-4db4-bb09-aaeca7cdc2db', 'ba759039-8cfa-4a5f-b105-d362fb3387ed', '9341efc0-deb4-465a-b517-7e66cf5df89d', 'a0a99d3c-b074-49f5-af90-05bd0e57e78c'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:43:59.214757 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:59.254136 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:43:59.287250 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:43:59.291597 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] Deleted volume successfully. May 28 21:43:59.362705 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-349c8ffb-fec6-44cc-a9d3-453d8341d7b3 tempest-VolumesTransfersV355Test-1658520325 None] Lock "b'cinder-62903144-b4f9-4936-8de2-f1d9f198c2bb-delete_volume'" released by "delete_volume" :: held 5.388s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:44:00.084794 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:44:00.104339 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Flow 'volume_create_manager' (4c646129-554e-425d-888b-adeb77deb9a6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:44:00.110753 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (932ccd31-9b35-470a-8ba3-03f86ca0a02a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:44:00.132390 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (932ccd31-9b35-470a-8ba3-03f86ca0a02a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:44:00Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesTransfersV355Test-Volume-1101851278',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=608ef588-9679-4581-9243-4aa1cccbb988,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='65965326d8ef4ea89a8a4245e147bee6',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:44:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:44:00Z,use_quota=True,user_id='a436e059356f4840ac977fcc5db9d5b3',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:44:00.132633 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (300a8eae-618e-45e9-a79f-88e4603a1dec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:44:00.135953 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (300a8eae-618e-45e9-a79f-88e4603a1dec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:44:00.141547 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6fa2cbee-7bfe-4048-9f79-aa92d4596101) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:44:00.145561 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6fa2cbee-7bfe-4048-9f79-aa92d4596101) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '608ef588-9679-4581-9243-4aa1cccbb988', 'volume_name': 'volume-608ef588-9679-4581-9243-4aa1cccbb988', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:44:00.150502 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1cf94dbc-0e09-49d6-a001-dcd31488c103) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:44:00.154908 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1cf94dbc-0e09-49d6-a001-dcd31488c103) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:44:00.159519 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aa2bbb14-6511-4746-a8ae-3f30ded3996e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:44:00.160561 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Volume 608ef588-9679-4581-9243-4aa1cccbb988: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-608ef588-9679-4581-9243-4aa1cccbb988', 'volume_size': 1} May 28 21:44:00.160943 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] creating volume 'volume-608ef588-9679-4581-9243-4aa1cccbb988' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:44:00.161677 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:44:00.781475 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:44:00.781863 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:44:00.785503 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aa2bbb14-6511-4746-a8ae-3f30ded3996e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-608ef588-9679-4581-9243-4aa1cccbb988', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:44:00.790064 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b4a387f4-74f0-4e22-879e-57eca8146993) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:44:00.793711 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=44,cinder:INSERT=7,cinder:UPDATE=34,cinder:DELETE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:00.899759 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Volume volume-608ef588-9679-4581-9243-4aa1cccbb988 (608ef588-9679-4581-9243-4aa1cccbb988): created successfully May 28 21:44:00.903420 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b4a387f4-74f0-4e22-879e-57eca8146993) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:44:00.908860 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Flow 'volume_create_manager' (4c646129-554e-425d-888b-adeb77deb9a6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:44:01.019953 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fc35ad8e-e262-40ce-9344-0c303a3cc484 tempest-VolumesTransfersV355Test-1675622206 None] Created volume successfully. May 28 21:44:01.089793 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:44:01.090476 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:44:01.114036 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:44:01.135346 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:44:02.722334 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] Lock "b'cinder-608ef588-9679-4581-9243-4aa1cccbb988-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:44:02.821141 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:44:02.956998 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:44:02.957433 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] Volume volume-608ef588-9679-4581-9243-4aa1cccbb988 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:44:03.104717 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] deleting rbd volume volume-608ef588-9679-4581-9243-4aa1cccbb988 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:44:08.025629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] Created reservations ['f9197174-4895-4462-88e5-694be498830f', 'aba4a1df-77d9-49a3-ad52-85a6825ed692', 'c10692ac-6cc1-408d-9bb7-4245b447a8a9', '7d4945e4-9394-4355-acbf-dfa246cb572e'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:44:08.207652 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:44:08.230289 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:44:08.252444 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:44:08.256480 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] Deleted volume successfully. May 28 21:44:08.315323 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-3c275b7c-af40-4d5a-8a9c-c9699c556398 tempest-VolumesTransfersV355Test-1658520325 None] Lock "b'cinder-608ef588-9679-4581-9243-4aa1cccbb988-delete_volume'" released by "delete_volume" :: held 5.593s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:44:18.259907 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=11,cinder:UPDATE=10,cinder:INSERT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:20.191871 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=22,cinder:SELECT=41,cinder:DELETE=3,cinder:INSERT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:44:50.181897 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:00.177301 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:01.091122 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:45:01.091782 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:45:01.113571 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:45:01.139339 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:45:20.183329 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:30.094264 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:45:30.105702 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-c1e9fc29-415f-4936-9eed-37dca461baa9 req-726684be-2920-48bd-8a52-dddb4f356c75 tempest-TestVolumeBootPattern-1994669512 None] Terminate volume connection completed successfully. May 28 21:45:30.106153 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-c1e9fc29-415f-4936-9eed-37dca461baa9 req-726684be-2920-48bd-8a52-dddb4f356c75 tempest-TestVolumeBootPattern-1994669512 None] Deleting attachment e1619758-598c-4403-822f-3be76a3403ca. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:45:31.365740 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-76158cb5-574c-466c-82f8-f0cc50204799-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:45:31.488340 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:45:31.646819 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:45:31.778397 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] deleting rbd volume volume-76158cb5-574c-466c-82f8-f0cc50204799 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:45:34.574022 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 21:45:34.671221 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] deleting parent snapshot volume-76158cb5-574c-466c-82f8-f0cc50204799.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 21:45:36.230183 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] Created reservations ['cc8a492d-1b08-4ca3-95b0-1cfef7f2b2d1', '2af3a1c6-8db5-4ed5-812a-7629d31cd684', '92d19c2c-3359-43ec-9f8a-509b0f9ffe5f', 'ac0b4c9b-5d1d-41a2-a490-06be9c52c8ee'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:45:36.407989 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:45:36.432763 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:45:36.531991 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:45:36.536312 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] Deleted volume successfully. May 28 21:45:36.620826 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-ddf75f9e-71b0-48ae-b447-9a89e723785e tempest-TestVolumeBootPattern-1994669512 None] Lock "b'cinder-76158cb5-574c-466c-82f8-f0cc50204799-delete_volume'" released by "delete_volume" :: held 5.255s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:45:50.193057 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=16,cinder:UPDATE=7,cinder:INSERT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:46:00.201674 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=21,cinder:UPDATE=12,cinder:INSERT=3,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:46:01.095419 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:46:01.096067 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:46:01.118217 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:46:01.141301 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:46:30.200098 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:46:40.203925 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:UPDATE=2,cinder:SELECT=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:00.208692 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:01.097108 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:47:01.097762 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:47:01.120401 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:47:01.141520 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:47:20.207499 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:40.206399 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:40.209552 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:47:50.243653 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:01.097083 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:48:01.097746 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:48:01.120462 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:48:01.142704 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:48:10.209934 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:20.216414 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:40.209916 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:48:50.219615 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:01.097996 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:49:01.098646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:49:01.124190 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:49:01.151577 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:49:10.217455 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:20.226981 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:49:30.233176 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:00.233629 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:01.096797 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:50:01.097454 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:50:01.119836 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:50:01.140972 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:50:10.236214 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:40.243215 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:50:50.241068 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:01.096089 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:51:01.096877 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:51:01.122268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:51:01.150859 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:51:10.241038 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:20.241797 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:50.241134 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:51:50.244127 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:52:01.096019 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:52:01.096701 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:52:01.117693 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:52:01.140009 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:52:20.252387 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:52:30.254789 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:53:00.245921 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:53:01.094694 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:53:01.096684 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:53:01.140416 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:53:01.177391 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:53:20.247884 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:53:40.246246 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:54:00.244187 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:54:01.092993 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:54:01.093669 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:54:01.115282 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:54:01.136435 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:54:20.251520 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:54:30.255039 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:54:50.258175 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:00.256831 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:01.091939 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:55:01.092594 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:55:01.115489 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:55:01.137450 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:55:09.883494 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:55:09.902682 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Flow 'volume_create_manager' (d0052198-6994-4b63-bb58-a12351daeac6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:55:09.908851 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (57a472d1-50c5-448c-830a-7b04ee279981) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:55:09.924568 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (57a472d1-50c5-448c-830a-7b04ee279981) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:55:09Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-ServerActionsTestJSON-volume-1347520597',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=aacef0b7-ccd9-44aa-b2dd-efab9ae27c17,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='77ee3a3a7c3c4288984836d79f10e732',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:55:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:55:10Z,use_quota=True,user_id='aa3da8047b124549a05e86b438789dfb',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:55:09.929776 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (36586610-6ac9-4ae9-8558-729be020e4cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:55:09.933746 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (36586610-6ac9-4ae9-8558-729be020e4cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:55:09.939540 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3eac3a56-4f33-4f26-b97c-a8581425fc7f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:55:09.943469 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (3eac3a56-4f33-4f26-b97c-a8581425fc7f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'volume_name': 'volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:55:09.948229 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (091c4a3e-874d-40f4-81e2-f70955c02b9c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:55:09.952311 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (091c4a3e-874d-40f4-81e2-f70955c02b9c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:55:09.957256 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7be47623-ec4b-4db5-b4c2-4bf78382e15e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:55:09.958332 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Volume aacef0b7-ccd9-44aa-b2dd-efab9ae27c17: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'volume_size': 1} May 28 21:55:09.958715 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] creating volume 'volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:55:09.959457 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:55:10.319490 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:55:10.319764 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:55:10.323862 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7be47623-ec4b-4db5-b4c2-4bf78382e15e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:55:10.328664 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d20721c9-7244-4b95-bad2-d710b7734a10) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:55:10.434397 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Volume volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17 (aacef0b7-ccd9-44aa-b2dd-efab9ae27c17): created successfully May 28 21:55:10.438067 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d20721c9-7244-4b95-bad2-d710b7734a10) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:55:10.443384 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Flow 'volume_create_manager' (d0052198-6994-4b63-bb58-a12351daeac6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:55:10.508751 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-fde2068f-bba0-4eb8-aac7-a4e97fa51730 tempest-ServerActionsTestJSON-1410661479 None] Created volume successfully. May 28 21:55:11.821413 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d8db4903-60e5-4bf7-9080-398b86c64ed9 req-cb77ff68-fbfb-4a56-b19a-c62a1e42386b tempest-ServerActionsTestJSON-1410661479 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:55:12.430503 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d8db4903-60e5-4bf7-9080-398b86c64ed9 req-cb77ff68-fbfb-4a56-b19a-c62a1e42386b tempest-ServerActionsTestJSON-1410661479 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.609s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:55:12.431259 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-d8db4903-60e5-4bf7-9080-398b86c64ed9 req-cb77ff68-fbfb-4a56-b19a-c62a1e42386b tempest-ServerActionsTestJSON-1410661479 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:55:12.499835 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-d8db4903-60e5-4bf7-9080-398b86c64ed9 req-cb77ff68-fbfb-4a56-b19a-c62a1e42386b tempest-ServerActionsTestJSON-1410661479 None] Connection info returned from driver {'name': 'volumes/volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5be6da2c-015f-4620-8b85-bfddc77bce0f'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:55:12.586485 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-d8db4903-60e5-4bf7-9080-398b86c64ed9 req-cb77ff68-fbfb-4a56-b19a-c62a1e42386b tempest-ServerActionsTestJSON-1410661479 None] attachment_update completed successfully. May 28 21:55:30.262878 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=23,cinder:UPDATE=5,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:30.686948 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=18,cinder:UPDATE=5 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:55:30.696014 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-b172d615-9875-4ecf-a5be-21e1e3e6b206 tempest-ServerActionsTestJSON-1410661479 None] Terminate volume connection completed successfully. May 28 21:55:30.696428 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-b172d615-9875-4ecf-a5be-21e1e3e6b206 tempest-ServerActionsTestJSON-1410661479 None] Deleting attachment 5be6da2c-015f-4620-8b85-bfddc77bce0f. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:55:33.834137 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-81429a22-c475-4ddc-a6c0-2af60854de6b tempest-ServerActionsTestJSON-1410661479 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:55:34.482824 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-81429a22-c475-4ddc-a6c0-2af60854de6b tempest-ServerActionsTestJSON-1410661479 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.647s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:55:34.483923 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-81429a22-c475-4ddc-a6c0-2af60854de6b tempest-ServerActionsTestJSON-1410661479 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:55:34.611819 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-81429a22-c475-4ddc-a6c0-2af60854de6b tempest-ServerActionsTestJSON-1410661479 None] Connection info returned from driver {'name': 'volumes/volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '929193a3-e318-4634-97a0-8c0f0de3138b'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:55:34.728478 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-3d4c2897-0aeb-4f93-87e3-1c7af7d2c804 req-81429a22-c475-4ddc-a6c0-2af60854de6b tempest-ServerActionsTestJSON-1410661479 None] attachment_update completed successfully. May 28 21:55:43.122189 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-e2830654-747a-4232-b9ff-f4fdf2fae001 tempest-ServerActionsTestJSON-1410661479 None] Terminate volume connection completed successfully. May 28 21:55:43.122771 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-e2830654-747a-4232-b9ff-f4fdf2fae001 tempest-ServerActionsTestJSON-1410661479 None] Deleting attachment 929193a3-e318-4634-97a0-8c0f0de3138b. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:55:46.026134 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-4ffedd56-1344-452e-8c30-6c42966a9e81 tempest-ServerActionsTestJSON-1410661479 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:55:46.683121 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-4ffedd56-1344-452e-8c30-6c42966a9e81 tempest-ServerActionsTestJSON-1410661479 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.657s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:55:46.683934 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-4ffedd56-1344-452e-8c30-6c42966a9e81 tempest-ServerActionsTestJSON-1410661479 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:55:46.770804 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-4ffedd56-1344-452e-8c30-6c42966a9e81 tempest-ServerActionsTestJSON-1410661479 None] Connection info returned from driver {'name': 'volumes/volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'aacef0b7-ccd9-44aa-b2dd-efab9ae27c17', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '7c1208ef-dcd4-46f6-b386-4942e199018b'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:55:46.868692 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-d856c8c0-bf06-4376-9b9b-4dee326e8c88 req-4ffedd56-1344-452e-8c30-6c42966a9e81 tempest-ServerActionsTestJSON-1410661479 None] attachment_update completed successfully. May 28 21:55:54.710244 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-04e6c901-0f13-4231-82aa-bdb57d18d3a3 req-3062debf-8dc3-4d2e-87e6-3717433b3e92 tempest-ServerActionsTestJSON-1410661479 None] Terminate volume connection completed successfully. May 28 21:55:54.710920 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-04e6c901-0f13-4231-82aa-bdb57d18d3a3 req-3062debf-8dc3-4d2e-87e6-3717433b3e92 tempest-ServerActionsTestJSON-1410661479 None] Deleting attachment 7c1208ef-dcd4-46f6-b386-4942e199018b. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:55:56.902027 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] Lock "b'cinder-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17-delete_volume'" acquired by "delete_volume" :: waited 0.010s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:55:56.996206 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:55:57.086437 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:55:57.086891 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] Volume volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:55:57.144156 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] deleting rbd volume volume-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:55:59.178079 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] Created reservations ['116b2f95-ab7a-41ae-840b-5f55607d85d9', '03623b3f-afb7-403b-a3f1-8772ea54d75a', 'c18ed5d2-cfca-40f7-8a33-0e8e09c45965', 'cba409d8-010e-473d-a766-ceebfcc154c0'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:55:59.299363 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:55:59.322194 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:55:59.344867 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:55:59.349129 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] Deleted volume successfully. May 28 21:55:59.450617 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-b0ac4a2b-b3eb-43b4-92c5-0e14277dda6d tempest-ServerActionsTestJSON-1410661479 None] Lock "b'cinder-aacef0b7-ccd9-44aa-b2dd-efab9ae27c17-delete_volume'" released by "delete_volume" :: held 2.549s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:56:10.266771 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=40,cinder:UPDATE=14,cinder:INSERT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:10.267435 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=49,cinder:UPDATE=11,cinder:INSERT=3,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:30.270237 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:56:50.269679 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:01.093985 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:57:01.094654 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:57:01.128588 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:57:01.163368 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:57:10.270970 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:30.272801 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:30.273515 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:50.266139 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:57:57.578045 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 21:57:57.597742 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Flow 'volume_create_manager' (f0c5db53-9c4a-4860-a62d-9ae2d50c2e24) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:57:57.604715 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (18ea46fe-846f-436b-ab7a-1e03fffd810a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:57:57.620726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (18ea46fe-846f-436b-ab7a-1e03fffd810a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T21:57:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TestMinimumBasicScenario-volume-1886635396',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=ef9bd7bc-d94a-4522-b99c-668994610269,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f1c74667a82a49b6b4334c53eabb842f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T21:57:57Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T21:57:57Z,use_quota=True,user_id='b99fe7794c22452484990b7e783f9a43',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:57:57.625941 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e480341a-b1fc-4a14-9bf0-e07703766758) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:57:57.629994 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e480341a-b1fc-4a14-9bf0-e07703766758) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:57:57.635432 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (742c043f-980d-40c0-b999-3d9aa59ccd28) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:57:57.639449 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (742c043f-980d-40c0-b999-3d9aa59ccd28) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'ef9bd7bc-d94a-4522-b99c-668994610269', 'volume_name': 'volume-ef9bd7bc-d94a-4522-b99c-668994610269', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:57:57.644203 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c1bc91c9-8f03-475d-9e6d-d880222cf413) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:57:57.647992 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (c1bc91c9-8f03-475d-9e6d-d880222cf413) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:57:57.653268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cc3844b6-18a9-4089-adae-4ffc68d9f2ea) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:57:57.654344 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Volume ef9bd7bc-d94a-4522-b99c-668994610269: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-ef9bd7bc-d94a-4522-b99c-668994610269', 'volume_size': 1} May 28 21:57:57.654727 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] creating volume 'volume-ef9bd7bc-d94a-4522-b99c-668994610269' {{(pid=135921) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1044}} May 28 21:57:57.655467 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:57:58.096251 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 21:57:58.096615 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 21:57:58.100263 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (cc3844b6-18a9-4089-adae-4ffc68d9f2ea) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-ef9bd7bc-d94a-4522-b99c-668994610269', 'volume_size': 1}' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:57:58.105538 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5f3fb4d9-fb4f-48d6-8d22-3f2fde8824de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 21:57:58.173391 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Volume volume-ef9bd7bc-d94a-4522-b99c-668994610269 (ef9bd7bc-d94a-4522-b99c-668994610269): created successfully May 28 21:57:58.176909 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5f3fb4d9-fb4f-48d6-8d22-3f2fde8824de) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 21:57:58.182283 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Flow 'volume_create_manager' (f0c5db53-9c4a-4860-a62d-9ae2d50c2e24) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 21:57:58.272779 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-0a3a058a-f25d-4cfc-8e78-a3fdcd960369 tempest-TestMinimumBasicScenario-1635957539 None] Created volume successfully. May 28 21:57:59.479097 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-70042a5b-bc25-455c-9d36-458da81b73ac req-e00a810b-9208-45c0-992b-0f7cefefbe0f tempest-TestMinimumBasicScenario-1635957539 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 21:58:00.175555 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-70042a5b-bc25-455c-9d36-458da81b73ac req-e00a810b-9208-45c0-992b-0f7cefefbe0f tempest-TestMinimumBasicScenario-1635957539 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.696s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 21:58:00.176395 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-70042a5b-bc25-455c-9d36-458da81b73ac req-e00a810b-9208-45c0-992b-0f7cefefbe0f tempest-TestMinimumBasicScenario-1635957539 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-ef9bd7bc-d94a-4522-b99c-668994610269', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'ef9bd7bc-d94a-4522-b99c-668994610269', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 21:58:00.282012 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-70042a5b-bc25-455c-9d36-458da81b73ac req-e00a810b-9208-45c0-992b-0f7cefefbe0f tempest-TestMinimumBasicScenario-1635957539 None] Connection info returned from driver {'name': 'volumes/volume-ef9bd7bc-d94a-4522-b99c-668994610269', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'ef9bd7bc-d94a-4522-b99c-668994610269', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c5c5ce30-e9f1-4bae-8956-090e9397ddf8'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 21:58:00.370179 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=20,cinder:UPDATE=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:58:00.394210 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-70042a5b-bc25-455c-9d36-458da81b73ac req-e00a810b-9208-45c0-992b-0f7cefefbe0f tempest-TestMinimumBasicScenario-1635957539 None] attachment_update completed successfully. May 28 21:58:01.098581 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:58:01.099447 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:58:01.124209 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:58:01.146602 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:58:20.273733 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=21,cinder:UPDATE=4,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:58:40.278162 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:58:40.279646 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:01.102059 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 21:59:01.102726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:59:01.124272 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:59:01.145746 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:59:10.277077 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:20.284173 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:22.792388 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-ed1810b8-c8f3-4c57-96b5-44a31c1dc24d req-8478e925-a611-445b-b07b-dc67c3c6121a tempest-TestMinimumBasicScenario-1635957539 None] Terminate volume connection completed successfully. May 28 21:59:22.796834 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ed1810b8-c8f3-4c57-96b5-44a31c1dc24d req-8478e925-a611-445b-b07b-dc67c3c6121a tempest-TestMinimumBasicScenario-1635957539 None] Deleting attachment c5c5ce30-e9f1-4bae-8956-090e9397ddf8. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 21:59:23.507020 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] Lock "b'cinder-ef9bd7bc-d94a-4522-b99c-668994610269-delete_volume'" acquired by "delete_volume" :: waited 0.009s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 21:59:23.595703 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:59:23.789232 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 21:59:23.789688 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] Volume volume-ef9bd7bc-d94a-4522-b99c-668994610269 is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 21:59:23.874940 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] deleting rbd volume volume-ef9bd7bc-d94a-4522-b99c-668994610269 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 21:59:26.162783 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] Created reservations ['9f4949b2-fda4-4ac6-a484-eed6dcd9543e', 'd672ef7c-e634-459d-a84a-f4e9c38295c2', 'cff9ecd1-1132-466b-81f9-e7b75767686b', '435ba7c1-0032-47cc-a198-e283c7e03717'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 21:59:26.317731 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:59:26.342226 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 21:59:26.366170 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 21:59:26.372977 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] Deleted volume successfully. May 28 21:59:26.435004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-84e11680-4a3c-46ac-873d-62cf478df9e8 tempest-TestMinimumBasicScenario-1635957539 None] Lock "b'cinder-ef9bd7bc-d94a-4522-b99c-668994610269-delete_volume'" released by "delete_volume" :: held 2.928s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 21:59:36.265962 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=12,cinder:INSERT=2,cinder:UPDATE=9 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 21:59:40.284880 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=22,cinder:UPDATE=9,cinder:INSERT=2,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:00:01.100917 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:00:01.101658 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:00:01.125324 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:00:01.149908 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:00:10.278938 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:00:30.271951 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:00:50.275432 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:01:01.105349 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:01:01.106289 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:01:01.131962 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:01:01.157745 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:01:10.279470 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:01:30.277141 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:01:30.279689 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:02:01.102998 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:02:01.103661 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:02:01.125125 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:02:01.146946 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:02:10.272093 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:02:10.272707 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:02:40.279064 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:02:50.271146 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:01.103295 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:03:01.103959 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:03:01.125127 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:03:01.146432 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:03:20.275045 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:40.280895 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:03:40.281823 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:00.284383 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:01.102078 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:04:01.102736 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:04:01.123844 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:04:01.146097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:04:10.291386 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:30.293158 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:04:40.294908 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:05:00.302285 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:05:01.102100 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:05:01.102768 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:05:01.124577 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:05:01.146630 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:05:20.316629 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:05:40.319728 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:00.322296 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:01.101660 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:06:01.102372 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:06:01.123511 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:06:01.145341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:06:10.317498 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:40.314964 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:06:50.311180 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:01.100992 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:07:01.101650 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:07:01.130092 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:07:01.152709 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:07:10.310982 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:30.315339 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:07:40.323410 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:08:01.098947 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:08:01.099652 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:08:01.120604 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:08:01.142260 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:08:10.320334 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:08:20.322730 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:08:40.323015 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:09:00.329082 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:09:01.098060 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:09:01.098726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:09:01.120828 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:09:01.145179 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:09:10.337367 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:09:40.332848 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:09:50.330439 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:01.099910 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:10:01.100569 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:10:01.123413 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:10:01.144484 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:10:10.332822 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:20.337987 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:10:50.336449 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:11:00.336212 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:11:01.100005 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:11:01.100701 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:11:01.122041 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:11:01.153004 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:11:30.340990 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:11:40.340235 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:00.352522 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:01.098550 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:12:01.099268 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:12:01.121322 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:12:01.143138 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:12:10.170571 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:10.243802 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=135921) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1350}} May 28 22:12:10.265069 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Flow 'volume_create_manager' (0688664d-e67c-4516-ab84-47b97ab45279) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 22:12:10.271532 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba8214c2-203d-44e2-82de-f2915d98cdba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 22:12:10.288059 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ba8214c2-203d-44e2-82de-f2915d98cdba) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=,attach_status='detached',availability_zone='nova',bootable=False,cluster=,cluster_name=None,consistencygroup=,consistencygroup_id=None,created_at=2022-05-28T22:12:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-239038212',ec2_id=None,encryption_key_id=None,glance_metadata=,group=,group_id=None,host='ncdv-master-1@ceph#ceph',id=afa9bd0d-561b-4f64-b2cc-9199f4a91324,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4ef8feba0c8e45fdb82f09eeab276061',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-05-28T22:12:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=,source_volid=None,status='creating',terminated_at=None,updated_at=2022-05-28T22:12:10Z,use_quota=True,user_id='31c2c3ecbc7148859a7c530deef8ab95',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(baa4f94a-73f9-4a6a-905a-4a2e4bce80ff),volume_type_id=baa4f94a-73f9-4a6a-905a-4a2e4bce80ff)' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 22:12:10.293432 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d5b55fe1-fc56-4789-af25-31fd8848a0fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 22:12:10.297525 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d5b55fe1-fc56-4789-af25-31fd8848a0fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 22:12:10.303264 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9d3c6df-6b04-4066-8e79-c13f9687bf2c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 22:12:10.382690 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b9d3c6df-6b04-4066-8e79-c13f9687bf2c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'volume_name': 'volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 22:12:10.387805 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (320ed218-b51a-4960-b0e7-b3dc85cabb03) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 22:12:10.391703 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (320ed218-b51a-4960-b0e7-b3dc85cabb03) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 22:12:10.396385 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (059df8b0-7390-478b-a566-741a4727ec43) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 22:12:10.397494 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Volume afa9bd0d-561b-4f64-b2cc-9199f4a91324: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': } May 28 22:12:10.398107 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Cloning afa9bd0d-561b-4f64-b2cc-9199f4a91324 from image 9586d0d3-c540-4636-bb36-06bb4b16c195 at location (None, None). {{(pid=135921) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1053}} May 28 22:12:10.406162 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" acquired by "_prepare_image_cache_entry" :: waited 0.007s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 22:12:10.558887 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 22, 12, 10, 411017)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 22:12:10.559341 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 22:12:10.559735 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Found cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:876}} May 28 22:12:10.564986 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Lock "b'cinder-9586d0d3-c540-4636-bb36-06bb4b16c195'" released by "_prepare_image_cache_entry" :: held 0.159s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 22:12:10.566192 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Attempting to retrieve cache entry for image = 9586d0d3-c540-4636-bb36-06bb4b16c195 on host ncdv-master-1@ceph#ceph. {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:817}} May 28 22:12:10.644952 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Found image-volume cache entry: {'id': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'volume_id': 'd155253c-0f01-4ae8-9e2c-11ffe940469e', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33), 'last_used': datetime.datetime(2022, 5, 28, 22, 12, 10, 570098)}. {{(pid=135921) get_entry /opt/stack/cinder/cinder/image/cache.py:76}} May 28 22:12:10.645374 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.image.cache [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Image-volume cache entry image_update_at = 2022-05-28 20:53:33+00:00, requested image updated_at = 2022-05-28 20:53:33+00:00. {{(pid=135921) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}} May 28 22:12:10.645766 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Creating from source image-volume d155253c-0f01-4ae8-9e2c-11ffe940469e {{(pid=135921) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:831}} May 28 22:12:10.658449 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:12:10.758895 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] creating snapshot='volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324.clone_snap' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:783}} May 28 22:12:12.256387 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] cloning 'volume-d155253c-0f01-4ae8-9e2c-11ffe940469e@volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324.clone_snap' to 'volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324' {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:789}} May 28 22:12:13.041325 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Volume volume-bdc702a5-c0fa-423b-b6bf-14c735b9af34.deleted is not a clone. {{(pid=135921) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1205}} May 28 22:12:13.042589 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:919}} May 28 22:12:13.042970 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=135921) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:928}} May 28 22:12:13.237515 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] clone created successfully {{(pid=135921) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:862}} May 28 22:12:13.238168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.volume_utils [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Marking volume afa9bd0d-561b-4f64-b2cc-9199f4a91324 as bootable. {{(pid=135921) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}} May 28 22:12:13.296487 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Copying metadata from image 9586d0d3-c540-4636-bb36-06bb4b16c195 to afa9bd0d-561b-4f64-b2cc-9199f4a91324. {{(pid=135921) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:450}} May 28 22:12:13.297052 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Creating volume glance metadata for volume afa9bd0d-561b-4f64-b2cc-9199f4a91324 backed by image 9586d0d3-c540-4636-bb36-06bb4b16c195 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=135921) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:673}} May 28 22:12:13.402338 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (059df8b0-7390-478b-a566-741a4727ec43) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'volume_size': 1, 'image_id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'a7d6b819de6044b98f2c78098814b8f2', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': '9586d0d3-c540-4636-bb36-06bb4b16c195', 'created_at': datetime.datetime(2022, 5, 28, 20, 53, 16, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 5, 28, 20, 53, 33, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/9586d0d3-c540-4636-bb36-06bb4b16c195/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': }' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 22:12:13.407001 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (81866e6a-91f0-4eb7-92bc-d1e09bbd2f29) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}} May 28 22:12:13.470336 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.flows.manager.create_volume [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Volume volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324 (afa9bd0d-561b-4f64-b2cc-9199f4a91324): created successfully May 28 22:12:13.473875 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (81866e6a-91f0-4eb7-92bc-d1e09bbd2f29) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=135921) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}} May 28 22:12:13.479202 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Flow 'volume_create_manager' (0688664d-e67c-4516-ab84-47b97ab45279) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=135921) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}} May 28 22:12:13.551004 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [None req-075ace0a-2543-4a6e-a081-88d9c6aac2c2 tempest-TestShelveInstance-491367012 None] Created volume successfully. May 28 22:12:21.556471 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-ffb191ee-957f-4437-a7c3-ad13ba39c31d req-77bfa77b-6510-48c4-a225-e2c34392a0ba tempest-TestShelveInstance-491367012 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}} May 28 22:12:22.298088 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_concurrency.processutils [req-ffb191ee-957f-4437-a7c3-ad13ba39c31d req-77bfa77b-6510-48c4-a225-e2c34392a0ba tempest-TestShelveInstance-491367012 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.741s {{(pid=135921) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}} May 28 22:12:22.298987 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-ffb191ee-957f-4437-a7c3-ad13ba39c31d req-77bfa77b-6510-48c4-a225-e2c34392a0ba tempest-TestShelveInstance-491367012 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': '09c9249f-256c-45da-b5e8-aced76ddcb19', 'volume_id': 'afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'discard': True}} {{(pid=135921) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1657}} May 28 22:12:22.399850 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-ffb191ee-957f-4437-a7c3-ad13ba39c31d req-77bfa77b-6510-48c4-a225-e2c34392a0ba tempest-TestShelveInstance-491367012 None] Connection info returned from driver {'name': 'volumes/volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'hosts': ['10.222.0.33'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'afa9bd0d-561b-4f64-b2cc-9199f4a91324', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '79944c8c-ed3e-4040-bea2-d078e1a610e0'} {{(pid=135921) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4828}} May 28 22:12:22.521761 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-ffb191ee-957f-4437-a7c3-ad13ba39c31d req-77bfa77b-6510-48c4-a225-e2c34392a0ba tempest-TestShelveInstance-491367012 None] attachment_update completed successfully. May 28 22:12:40.344268 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=34,cinder:UPDATE=5,cinder:INSERT=6 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:12:40.345814 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=27,cinder:UPDATE=7,cinder:INSERT=6,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:13:01.100044 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:13:01.100909 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:13:01.124490 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:13:01.146568 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:13:10.343148 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:13:20.348929 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:13:30.356814 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:13:50.355099 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:14:00.352161 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:14:01.102044 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:14:01.102733 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:14:01.123720 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:14:01.145819 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:14:30.356046 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:14:40.356116 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:15:01.101815 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:15:01.102531 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:15:01.124473 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:15:01.146991 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:15:10.355507 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:15:10.357601 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:15:30.358058 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:15:50.357630 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=9,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:01.103169 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:16:01.103852 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:16:01.126201 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:16:01.147810 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:16:09.818208 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:09.820667 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-c7efe161-a698-4aa8-814d-aa77d4f96234 tempest-TestShelveInstance-491367012 None] Terminate volume connection completed successfully. May 28 22:16:09.821074 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.manager [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-c7efe161-a698-4aa8-814d-aa77d4f96234 tempest-TestShelveInstance-491367012 None] Deleting attachment 79944c8c-ed3e-4040-bea2-d078e1a610e0. {{(pid=135921) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4959}} May 28 22:16:10.124558 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] Lock "b'cinder-afa9bd0d-561b-4f64-b2cc-9199f4a91324-delete_volume'" acquired by "delete_volume" :: waited 0.014s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:186}} May 28 22:16:10.289403 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:16:10.469014 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] volume has no backup snaps {{(pid=135921) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1177}} May 28 22:16:10.561832 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] deleting rbd volume volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324 {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1331}} May 28 22:16:13.472299 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] volume is a clone so cleaning references {{(pid=135921) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1352}} May 28 22:16:13.609903 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] deleting parent snapshot volume-afa9bd0d-561b-4f64-b2cc-9199f4a91324.clone_snap {{(pid=135921) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1243}} May 28 22:16:15.621016 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.quota [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] Created reservations ['64ee9ed0-7df3-46a3-a7b0-a2ce8c2c5908', 'e123e916-d3fa-4c2e-9c5b-62ee42e11e82', '586016b7-c826-4dff-903b-c4dd2cc210fe', 'd0b95c30-5fa6-4dac-a69f-8a2effbba396'] {{(pid=135921) reserve /opt/stack/cinder/cinder/quota.py:755}} May 28 22:16:15.832204 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:16:15.857236 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:16:15.885101 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:16:15.890655 ncdv-master-1 cinder-volume[135921]: INFO cinder.volume.manager [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] Deleted volume successfully. May 28 22:16:15.972072 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.coordination [req-2d7e9afc-9280-42fe-a36b-1655de66474c req-7e6e8158-c997-4282-89b9-109cc658fbdf tempest-TestShelveInstance-491367012 None] Lock "b'cinder-afa9bd0d-561b-4f64-b2cc-9199f4a91324-delete_volume'" released by "delete_volume" :: held 5.848s {{(pid=135921) _synchronized /opt/stack/cinder/cinder/coordination.py:198}} May 28 22:16:25.896249 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=19,cinder:UPDATE=11,cinder:INSERT=2,cinder:DELETE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:40.570446 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=19,cinder:INSERT=2,cinder:UPDATE=9 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:16:50.573630 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:17:01.104316 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:17:01.104989 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:17:01.191701 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:17:01.214927 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:17:20.572960 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:17:30.576840 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:17:50.577980 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:18:01.101886 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:18:01.102608 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:18:01.123629 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:18:01.144942 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:18:10.579936 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:18:30.577702 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:18:50.582280 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:01.100543 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:19:01.101266 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:19:01.137221 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:19:01.160107 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:19:10.576090 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:20.582884 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:19:40.578041 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:20:00.586150 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:20:01.104993 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:20:01.105646 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:20:01.128203 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:20:01.150159 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:20:20.589198 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:20:40.587983 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:00.594139 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:01.105556 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:21:01.106177 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:21:01.142685 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:21:01.184071 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:21:30.591880 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:21:40.593346 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:22:00.596894 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:22:01.109427 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:22:01.110119 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:22:01.133326 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:22:01.155470 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:22:10.599467 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:22:40.592984 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:22:50.600757 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:23:01.110828 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:23:01.111490 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:23:01.134079 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:23:01.155297 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:23:20.598648 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:23:40.590917 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:24:00.599858 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:24:01.110221 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:24:01.110880 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:24:01.156200 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:24:01.177907 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:24:20.598252 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:24:30.605748 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:24:50.603311 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:25:00.595900 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:25:01.113964 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:25:01.114614 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:25:01.136639 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:25:01.158097 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:25:10.604331 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:25:30.600821 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:25:40.601035 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:26:00.602817 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:26:01.113288 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:26:01.113965 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:26:01.135788 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:26:01.157300 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:26:10.599659 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:26:30.607887 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:26:50.608544 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:27:01.111960 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:27:01.112606 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:27:01.134726 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:27:01.156168 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:27:10.609371 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:27:20.616156 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:27:50.611233 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=8,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:28:00.607844 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:28:01.111413 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:28:01.112074 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:28:01.133696 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:28:01.155178 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:28:20.611355 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:28:40.616619 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=6,cinder:UPDATE=4 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:29:00.616117 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:29:00.620175 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 28 22:29:01.110577 ncdv-master-1 cinder-volume[135921]: DEBUG oslo_service.periodic_task [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=135921) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}} May 28 22:29:01.111254 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:29:01.132716 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.volume.drivers.rbd [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=135921) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:527}} May 28 22:29:01.154031 ncdv-master-1 cinder-volume[135921]: DEBUG cinder.manager [None req-975bf044-f9ff-4d27-89c8-c88539e7524e None None] Notifying Schedulers of capabilities ... {{(pid=135921) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}} May 28 22:29:20.624789 ncdv-master-1 cinder-volume[135921]: DEBUG dbcounter [-] [135921] Writing DB stats cinder:SELECT=2 {{(pid=135921) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}