-- Logs begin at Sat 2022-07-02 20:01:19 UTC, end at Sat 2022-07-02 22:26:38 UTC. --
Jul 02 20:50:54.086308 ncdv-master-1 systemd[1]: Started Devstack devstack@c-vol.service.
Jul 02 20:50:57.614574 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_concurrency.lockutils [-] Acquired lock "singleton_lock" {{(pid=123840) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:294}}
Jul 02 20:50:57.623254 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_concurrency.lockutils [-] Releasing lock "singleton_lock" {{(pid=123840) lock /usr/local/lib/python3.8/dist-packages/oslo_concurrency/lockutils.py:312}}
Jul 02 20:50:58.064943 ncdv-master-1 cinder-volume[123840]: INFO dbcounter [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Registered counter for database cinder
Jul 02 20:50:58.083766 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_db.sqlalchemy.engines [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123840) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jul 02 20:50:58.090825 ncdv-master-1 cinder-volume[123840]: DEBUG dbcounter [-] [123840] Writer thread running {{(pid=123840) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}}
Jul 02 20:50:58.399561 ncdv-master-1 cinder-volume[123840]: INFO cinder.rpc [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Automatically selected cinder-scheduler objects version 1.39 as minimum service version.
Jul 02 20:50:58.404756 ncdv-master-1 cinder-volume[123840]: INFO cinder.rpc [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Automatically selected cinder-scheduler RPC version 3.12 as minimum service version.
Jul 02 20:50:58.413645 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_db.sqlalchemy.engines [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123840) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jul 02 20:50:58.487219 ncdv-master-1 cinder-volume[123840]: INFO cinder.volume.manager [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Determined volume DB was empty at startup.
Jul 02 20:50:58.487754 ncdv-master-1 cinder-volume[123840]: DEBUG cinder.volume.manager [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Cinder Volume DB check: vol_db_empty=True {{(pid=123840) __init__ /opt/stack/cinder/cinder/volume/manager.py:290}}
Jul 02 20:50:58.497131 ncdv-master-1 cinder-volume[123840]: INFO cinder.volume.manager [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Service not found for updating active_backend_id, assuming default for driver init.
Jul 02 20:50:58.497960 ncdv-master-1 cinder-volume[123840]: WARNING stevedore.named [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager
Jul 02 20:50:58.498397 ncdv-master-1 cinder-volume[123840]: WARNING castellan.key_manager [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager'
Jul 02 20:50:58.499566 ncdv-master-1 cinder-volume[123840]: WARNING cinder.keymgr.conf_key_mgr [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] This key manager is insecure and is not recommended for production deployments: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager'
Jul 02 20:50:58.515856 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_db.sqlalchemy.engines [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123840) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jul 02 20:50:58.520885 ncdv-master-1 cinder-volume[123840]: INFO cinder.rpc [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Automatically selected cinder-volume objects version 1.39 as minimum service version.
Jul 02 20:50:58.525477 ncdv-master-1 cinder-volume[123840]: INFO cinder.rpc [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Automatically selected cinder-volume RPC version 3.18 as minimum service version.
Jul 02 20:50:58.526187 ncdv-master-1 cinder-volume[123840]: WARNING stevedore.named [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Could not load cinder.keymgr.conf_key_mgr.ConfKeyManager
Jul 02 20:50:58.526588 ncdv-master-1 cinder-volume[123840]: WARNING castellan.key_manager [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Deprecation Warning : cinder.keymgr.conf_key_mgr.ConfKeyManager is not a stevedore based driver, trying to load it as a class: stevedore.exception.NoMatches: No 'castellan.drivers' driver found, looking for 'cinder.keymgr.conf_key_mgr.ConfKeyManager'
Jul 02 20:50:58.528397 ncdv-master-1 cinder-volume[123840]: INFO cinder.volume.manager [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Image-volume cache enabled for host ncdv-master-1@ceph.
Jul 02 20:50:58.535234 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_db.sqlalchemy.engines [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=123840) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jul 02 20:50:58.557532 ncdv-master-1 cinder-volume[123840]: INFO oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Starting 1 workers
Jul 02 20:50:58.563324 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Started child 124013 {{(pid=123840) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Jul 02 20:50:58.565172 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Full set of CONF: {{(pid=123840) wait /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:649}}
Jul 02 20:50:58.565818 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ******************************************************************************** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}}
Jul 02 20:50:58.566400 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Configuration options gathered from: {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}}
Jul 02 20:50:58.567026 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] command line args: ['--config-file', '/etc/cinder/cinder.conf'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}}
Jul 02 20:50:58.568062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] config files: ['/etc/cinder/cinder.conf'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}}
Jul 02 20:50:58.568625 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ================================================================================ {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}}
Jul 02 20:50:58.569385 ncdv-master-1 cinder-volume[124013]: INFO cinder.service [-] Starting cinder-volume node (version 20.1.0)
Jul 02 20:50:58.570404 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] allow_availability_zone_fallback = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.571065 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] allow_compression_on_image_upload = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.571800 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] allowed_direct_url_schemes     = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.582907 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] api_rate_limit                 = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.584024 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] auth_strategy                  = keystone {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.584721 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] az_cache_duration              = 3600 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.585379 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_availability_zone      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.586007 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_host                   = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.586621 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_name                   = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.587407 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_stats_polling_interval = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.588027 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backup_api_class               = cinder.backup.api.API {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.588662 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backup_manager                 = cinder.backup.manager.BackupManager {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.589287 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backup_name_template           = backup-%s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.589901 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backup_use_same_host           = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.590511 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backup_use_temp_snapshot       = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.591114 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] chap_password                  = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.591858 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] chap_username                  =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.593628 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] chiscsi_conf                   = /etc/chelsio-iscsi/chiscsi.conf {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.594331 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_internal_tenant_project_id = 001bf63308784f4eb62e64d6f82c8354 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.594943 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_internal_tenant_user_id = f9ea416fd4d4420b9112507ef1a68bf7 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.595505 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cloned_volume_same_az          = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.598862 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cluster                        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.598862 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] compression_format             = gzip {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.598862 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] compute_api_class              = cinder.compute.nova.API {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.598862 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] config_dir                     = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.598862 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] config_file                    = ['/etc/cinder/cinder.conf'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.598862 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] config_source                  = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.599375 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] consistencygroup_api_class     = cinder.consistencygroup.api.API {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.599375 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] control_exchange               = openstack {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.599375 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] debug                          = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.599375 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] default_availability_zone      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.599852 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] default_group_type             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.600470 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] default_log_levels             = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.601910 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] default_volume_type            = ceph {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.601910 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] driver_client_cert             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.601910 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] driver_client_cert_key         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.602184 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] driver_data_namespace          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.602716 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] driver_ssl_cert_path           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.602843 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] driver_ssl_cert_verify         = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.603440 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] driver_use_ssl                 = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.603641 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Starting volume driver RBDDriver (1.2.0)
Jul 02 20:50:58.603899 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] enable_force_upload            = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.607975 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] enable_new_services            = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.607975 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] enable_unsupported_driver      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608376 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:58.608485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] enabled_backends               = ['ceph'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] enforce_multipath_for_image_xfer = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] filter_function                = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_api_insecure            = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_api_servers             = ['http://10.222.0.43/image'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_api_ssl_compression     = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608914 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_ca_certificates_file    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608914 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_catalog_info            = image:glance:publicURL {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.608914 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_certfile                = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.609117 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_core_properties         = ['checksum', 'container_format', 'disk_format', 'image_name', 'image_id', 'min_disk', 'min_ram', 'name', 'size'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.609525 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_keyfile                 = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.609744 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_num_retries             = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.610063 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] glance_request_timeout         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.611474 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] goodness_function              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.611474 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] graceful_shutdown_timeout      = 5 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.611474 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] group_api_class                = cinder.group.api.API {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.611474 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] host                           = ncdv-master-1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.612312 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iet_conf                       = /etc/iet/ietd.conf {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.612460 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_compress_on_upload       = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.616062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_conversion_address_space_limit = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.616062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_conversion_cpu_limit     = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.616062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_conversion_dir           = /opt/stack/data/cinder/conversion {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.616062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_conversion_disable       = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.616062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_upload_use_cinder_backend = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.616062 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_upload_use_internal_tenant = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_volume_cache_enabled     = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_volume_cache_max_count   = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] image_volume_cache_max_size_gb = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] init_host_max_objects_retrieval = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] instance_format                = [instance: %(uuid)s]  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] instance_uuid_format           = [instance: %(uuid)s]  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.617627 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iscsi_iotype                   = fileio {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.618709 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iscsi_secondary_ip_addresses   = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.618709 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iscsi_target_flags             =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.618990 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iscsi_write_cache              = on {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.619151 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iser_helper                    = tgtadm {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.620139 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iser_ip_address                = 10.222.0.43 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.620139 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iser_port                      = 3260 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.620462 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] iser_target_prefix             = iqn.2010-10.org.openstack: {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.622432 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_config_append              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.622432 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_date_format                = %Y-%m-%d %H:%M:%S {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.622432 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_dir                        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.622432 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_file                       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.622432 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_options                    = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.623220 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_rotate_interval            = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.623220 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_rotate_interval_type       = days {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.623561 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] log_rotation_type              = none {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.623867 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] logging_context_format_string  = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.624305 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] logging_debug_format_suffix    = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.624693 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] logging_default_format_string  = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.632067 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] logging_exception_prefix       = ERROR %(name)s %(instance)s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.632067 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] logging_user_identity_format   = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.632067 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] max_age                        = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.632067 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] max_logfile_count              = 30 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.632067 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] max_logfile_size_mb            = 200 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.632067 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] max_over_subscription_ratio    = 20.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] message_reap_interval          = 86400 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] message_ttl                    = 2592000 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] migration_create_volume_timeout_secs = 300 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] monkey_patch                   = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] monkey_patch_modules           = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] my_ip                          = 10.222.0.43 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.633213 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] no_snapshot_gb_quota           = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] num_iser_scan_tries            = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] num_shell_tries                = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] num_volume_device_scan_tries   = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] nvmet_ns_id                    = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] nvmet_port_id                  = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_max_limit                = 1000 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.634325 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_volume_ext_list          = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.635397 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_volume_extension         = ['cinder.api.contrib.standard_extensions'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.635397 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_volume_listen            = 0.0.0.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.635397 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_volume_listen_port       = 8776 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.635397 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_volume_use_ssl           = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.635951 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] osapi_volume_workers           = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.635951 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] per_volume_size_limit          = -1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.636334 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] periodic_fuzzy_delay           = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.636579 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] periodic_interval              = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.638601 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] project_id_regex               = [0-9a-f\-]+ {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.638601 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] public_endpoint                = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.639183 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] publish_errors                 = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.639836 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_backup_gigabytes         = 1000 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.640469 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_backups                  = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.640991 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_consistencygroups        = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.641833 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_driver                   = cinder.quota.DbQuotaDriver {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.642361 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_gigabytes                = 1000 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.642859 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_groups                   = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.643396 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_snapshots                = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.644023 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] quota_volumes                  = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.644561 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] rate_limit_burst               = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.645072 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] rate_limit_except_level        = CRITICAL {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.645586 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] rate_limit_interval            = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.646095 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] reinit_driver_count            = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.646606 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] replication_device             = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.647838 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] report_discard_supported       = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.648665 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] report_interval                = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.649334 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] reservation_clean_interval     = 86400 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.649872 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] reservation_expire             = 86400 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.650414 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] reserved_percentage            = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.650952 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] resource_query_filters_file    = /etc/cinder/resource_filters.json {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.651477 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] rootwrap_config                = /etc/cinder/rootwrap.conf {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.651677 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:58.652215 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] rpc_response_timeout           = 120 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.652754 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] run_external_periodic_tasks    = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.653128 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] scheduler_manager              = cinder.scheduler.manager.SchedulerManager {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.653492 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] scst_target_driver             = iscsi {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.653930 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] scst_target_iqn_name           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.654253 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_down_time              = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.654648 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] snapshot_name_template         = snapshot-%s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.655022 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] snapshot_same_host             = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.655368 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] split_loggers                  = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.655658 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] state_path                     = /opt/stack/data/cinder {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.656014 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] storage_availability_zone      = nova {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.656418 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] storage_protocol               = iSCSI {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.656775 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] syslog_log_facility            = LOG_USER {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.657140 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] target_helper                  = lioadm {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.657610 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] target_ip_address              = 10.222.0.43 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.657978 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] target_port                    = 3260 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.658333 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] target_prefix                  = iqn.2010-10.org.openstack: {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.658793 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] target_protocol                = iscsi {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.659207 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] trace_flags                    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.659575 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] transfer_api_class             = cinder.transfer.api.API {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.660133 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] transport_url                  = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.660484 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] until_refresh                  = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.660831 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_chap_auth                  = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.661256 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_default_quota_class        = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.661619 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_eventlog                   = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.661961 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_journal                    = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.662301 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_json                       = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.662641 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_multipath_for_image_xfer   = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.662983 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_stderr                     = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.663376 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] use_syslog                     = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.663764 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] verify_glance_signatures       = enabled {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.664241 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_api_class               = cinder.volume.api.API {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.664696 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_backend_name            = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.665102 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_clear                   = zero {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.665477 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_clear_ionice            = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.665812 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_clear_size              = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.666168 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_copy_blkio_cgroup_name  = cinder-volume-copy {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.666513 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_copy_bps_limit          = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.666866 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_dd_blocksize            = 1M {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.667218 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_manager                 = cinder.volume.manager.VolumeManager {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.667589 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_name_template           = volume-%s {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.667963 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_service_inithost_offload = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.668337 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volume_usage_audit_period      = month {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.668771 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] volumes_dir                    = /opt/stack/data/cinder/volumes {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.669112 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] watch_log_file                 = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.669448 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] zoning_mode                    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}}
Jul 02 20:50:58.669941 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.670341 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_versionedobjects.fatal_exception_format_errors = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.670733 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_concurrency.disable_process_locking = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.671119 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_concurrency.lock_path     = /opt/stack/data/cinder {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.671517 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.671853 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_metrics.metrics_enabled = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.672260 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_metrics.metrics_process_name =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.672622 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.673326 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.673758 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.admin_password = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.674113 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.admin_tenant_name = admin {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.674463 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.admin_token = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.674824 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.admin_user  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.675212 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_admin_prefix =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.675580 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_host   = 127.0.0.1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.675941 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_port   = 35357 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.678364 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_protocol = https {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.678708 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_section = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.679191 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_type   = password {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.679479 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_uri    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.679833 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.auth_version = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.681578 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.cache       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.681975 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.cafile      = /opt/stack/data/ca-bundle.pem {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.682554 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.certfile    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.683717 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.delay_auth_decision = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.684130 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.enforce_token_bind = permissive {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.684681 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.http_connect_timeout = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.685196 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.http_request_max_retries = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.685711 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.identity_uri = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.686313 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.include_service_catalog = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.686825 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.insecure    = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.687333 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.interface   = public {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.687853 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.keyfile     = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.688377 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.688929 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.689453 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.689818 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.690174 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.690525 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_secret_key = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.691059 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_security_strategy = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.691421 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.693096 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.memcached_servers = ['localhost:11211'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.693461 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.region_name = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.693870 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.service_token_roles = ['service'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.694232 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.service_token_roles_required = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.694584 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.service_type = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.694933 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.token_cache_time = 300 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.695468 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] keystone_authtoken.www_authenticate_uri = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.696250 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.backend                  = dogpile.cache.null {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.696780 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.backend_argument         = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.697370 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.config_prefix            = cache.oslo {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.697941 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.dead_timeout             = 60.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.698166 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:58.698610 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.debug_cache_backend      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.699101 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.enable_retry_client      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.699586 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.enable_socket_keepalive  = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.700218 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.enabled                  = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.700731 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.expiration_time          = 600 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.701228 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.hashclient_retry_attempts = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.701710 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.hashclient_retry_delay   = 1.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.702214 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_dead_retry      = 300 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.702733 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_pool_connection_get_timeout = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.703239 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_pool_flush_on_reconnect = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.703772 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_pool_maxsize    = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.704292 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_pool_unused_timeout = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.704812 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_servers         = ['localhost:11211'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.705314 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.memcache_socket_timeout  = 1.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.705812 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.proxies                  = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.706349 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.retry_attempts           = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.706865 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.retry_delay              = 0.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.707341 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.socket_keepalive_count   = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.707834 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.socket_keepalive_idle    = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.708247 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.socket_keepalive_interval = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.708596 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.tls_allowed_ciphers      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.708939 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.tls_cafile               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.709290 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.tls_certfile             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.709634 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.tls_enabled              = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.709993 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cache.tls_keyfile              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.711539 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] coordination.backend_url       = etcd3+http://10.222.0.43:2379 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.712424 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.backend               = sqlalchemy {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.712710 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.connection            = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.713280 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.connection_debug      = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.713647 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.connection_parameters =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.714034 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.connection_recycle_time = 3600 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.714397 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.connection_trace      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.714745 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.db_inc_retry_interval = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.715190 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.db_max_retries        = 20 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.715883 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.db_max_retry_interval = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.716520 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.db_retry_interval     = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.717053 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.max_overflow          = 50 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.717399 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.max_pool_size         = 5 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.717907 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.max_retries           = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.718413 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.mysql_enable_ndb      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.718930 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.mysql_sql_mode        = TRADITIONAL {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.719433 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.pool_timeout          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.719794 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.retry_interval        = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.720186 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.slave_connection      = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.720549 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.sqlite_synchronous    = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.720893 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] database.use_db_reconnect      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.721451 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.enforce_new_defaults = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.721964 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.enforce_scope      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.722488 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.policy_default_rule = default {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.723038 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.policy_dirs        = ['policy.d'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.723636 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.policy_file        = policy.yaml {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.724136 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.724528 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.725031 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.remote_ssl_client_crt_file = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.725395 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.remote_ssl_client_key_file = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.725762 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.726186 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.connection_string     = messaging:// {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.726475 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.enabled               = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.726835 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.es_doc_type           = notification {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.727187 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.es_scroll_size        = 10000 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.727576 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.es_scroll_time        = 2m {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.727922 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.filter_error_trace    = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.728314 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.hmac_keys             = SECRET_KEY {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.728676 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.sentinel_service_name = mymaster {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.729050 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.socket_timeout        = 0.1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.729409 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] profiler.trace_sqlalchemy      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.729825 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] privsep_osbrick.capabilities   = [21] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.730195 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] privsep_osbrick.group          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.730535 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] privsep_osbrick.helper_command = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.730913 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] privsep_osbrick.logger_name    = oslo_privsep.daemon {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.731250 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] privsep_osbrick.thread_pool_size = 4 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.731606 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] privsep_osbrick.user           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.731991 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_sys_admin.capabilities  = [0, 1, 2, 3, 12, 21] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.732619 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_sys_admin.group         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.732962 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_sys_admin.helper_command = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.733320 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_sys_admin.logger_name   = oslo_privsep.daemon {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.733848 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_sys_admin.thread_pool_size = 4 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.735738 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] cinder_sys_admin.user          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.736341 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.auth_section      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.737057 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.auth_type         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.737783 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.cafile            = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.738304 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.certfile          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.738696 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.collect_timing    = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.739049 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.insecure          = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.739395 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.keyfile           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.739754 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.send_service_user_token = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.740315 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.split_loggers     = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.740931 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] service_user.timeout           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.741786 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.backend_availability_zone = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.742393 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.backend_native_threads_pool_size = 20 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.742986 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.backup_use_temp_snapshot = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.743566 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.chap_password = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.744138 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.chap_username =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.744678 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.chiscsi_conf  = /etc/chelsio-iscsi/chiscsi.conf {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.745211 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.driver [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Driver hasn't implemented _init_vendor_properties()
Jul 02 20:50:58.745351 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.deferred_deletion_delay = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.745783 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.driver [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.94, 'free_capacity_gb': 19.94, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:2b29f0f5-9ab4-46b2-a4f8-882559fe3d40:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=124013) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}}
Jul 02 20:50:58.746039 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.deferred_deletion_purge_interval = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.746555 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.driver_client_cert = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.747108 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.driver_client_cert_key = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.747723 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.driver_data_namespace = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.749472 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.driver_ssl_cert_path = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.750088 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.driver_ssl_cert_verify = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.750730 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.driver_use_ssl = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.751231 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.enable_deferred_deletion = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.751750 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.enable_unsupported_driver = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.752286 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.enforce_multipath_for_image_xfer = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.752998 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.extra_capabilities = {} {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.753490 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.filter_function = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.753991 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.goodness_function = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.754493 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iet_conf      = /etc/iet/ietd.conf {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.755001 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.image_upload_use_cinder_backend = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.755500 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.image_upload_use_internal_tenant = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.756009 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.image_volume_cache_enabled = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.756533 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.image_volume_cache_max_count = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.757049 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.image_volume_cache_max_size_gb = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.757572 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iscsi_iotype  = fileio {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.758080 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iscsi_secondary_ip_addresses = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.758677 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iscsi_target_flags =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.759206 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iscsi_write_cache = on {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.759699 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iser_helper   = tgtadm {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.760435 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iser_ip_address = 10.222.0.43 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.761122 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iser_port     = 3260 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.761625 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.iser_target_prefix = iqn.2010-10.org.openstack: {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.762203 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.max_over_subscription_ratio = 20.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.763380 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.num_iser_scan_tries = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.763909 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.num_shell_tries = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.764292 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_db.sqlalchemy.engines [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=124013) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jul 02 20:50:58.764622 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.num_volume_device_scan_tries = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.765136 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.nvmet_ns_id   = 10 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.765650 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.nvmet_port_id = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.766170 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rados_connect_timeout = -1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.766687 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rados_connection_interval = 5 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.767198 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rados_connection_retries = 3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.767736 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_ceph_conf =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.768293 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_cluster_name = ceph {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.768804 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_exclusive_cinder_pool = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.769018 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writer thread running {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}}
Jul 02 20:50:58.769484 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_flatten_volume_from_snapshot = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.770040 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_max_clone_depth = 5 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.770551 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_pool      = rbd {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.771060 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_secret_uuid = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.771639 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_store_chunk_size = 4 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.772710 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.rbd_user      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.773238 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.replication_connect_timeout = 5 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.773796 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.replication_device = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.774339 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.report_discard_supported = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.774849 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.report_dynamic_total_capacity = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.775354 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.reserved_percentage = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.775872 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.scst_target_driver = iscsi {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.776387 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.scst_target_iqn_name = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.776922 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.storage_protocol = iSCSI {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.777489 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.suppress_requests_ssl_warnings = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.778025 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.target_helper = tgtadm {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.778701 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.target_ip_address = 10.222.0.43 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.779196 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.target_port   = 3260 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.779696 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.target_prefix = iqn.2010-10.org.openstack: {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.780229 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.target_protocol = iscsi {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.780740 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.trace_flags   = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.781230 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.unique_fqdn_network = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.781722 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.use_chap_auth = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.782211 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.use_multipath_for_image_xfer = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.782690 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_backend_name = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.783199 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_clear  = zero {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.783688 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_clear_ionice = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.784214 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_clear_size = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.784772 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_copy_blkio_cgroup_name = cinder-volume-copy {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.785251 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_copy_bps_limit = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.785756 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_dd_blocksize = 1M {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.786246 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volume_driver = cinder.volume.drivers.lvm.LVMVolumeDriver {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.786827 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] backend_defaults.volumes_dir   = /opt/stack/data/cinder/volumes {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.787371 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_reports.file_event_handler = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.787868 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_reports.file_event_handler_interval = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.788394 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_reports.log_dir           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.788924 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.backend_availability_zone = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.789413 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.backend_host              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.789979 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.backend_native_threads_pool_size = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.790473 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.backup_use_temp_snapshot  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.790967 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.chap_password             = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.791489 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.chap_username             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.791861 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.chiscsi_conf              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.792244 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.deferred_deletion_delay   = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.792824 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.deferred_deletion_purge_interval = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.793216 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.driver_client_cert        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.793637 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.driver_client_cert_key    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.793979 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.driver_data_namespace     = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.794329 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.driver_ssl_cert_path      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.794777 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.driver_ssl_cert_verify    = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.795163 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.driver_use_ssl            = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.795691 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.enable_deferred_deletion  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.796161 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.enable_unsupported_driver = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.796643 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.enforce_multipath_for_image_xfer = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.797000 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.extra_capabilities        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.797378 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.filter_function           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.797754 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.goodness_function         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.798600 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iet_conf                  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.799173 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.image_upload_use_cinder_backend = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.799765 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.image_upload_use_internal_tenant = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.800463 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.image_volume_cache_enabled = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.800965 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.image_volume_cache_max_count = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.801456 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.image_volume_cache_max_size_gb = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.801942 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iscsi_iotype              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.802543 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iscsi_secondary_ip_addresses = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.803041 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iscsi_target_flags        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.803529 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iscsi_write_cache         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.804190 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iser_helper               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.804568 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iser_ip_address           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.804954 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iser_port                 = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.805302 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.iser_target_prefix        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.805680 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.max_over_subscription_ratio = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.806024 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.num_iser_scan_tries       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.806391 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.num_shell_tries           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.806739 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.num_volume_device_scan_tries = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.807235 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.nvmet_ns_id               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.807605 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.nvmet_port_id             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.807973 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rados_connect_timeout     = -1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.808860 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rados_connection_interval = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.809272 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rados_connection_retries  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.809698 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_ceph_conf             = /etc/ceph/ceph.conf {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.810098 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_cluster_name          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.810501 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_exclusive_cinder_pool = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.810938 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_flatten_volume_from_snapshot = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.811354 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_max_clone_depth       = 5 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.811764 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_pool                  = volumes {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.812229 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_secret_uuid           = bf7f2f0c-b9c4-4629-91bb-9884bc3ae388 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.812601 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.db.sqlalchemy.api [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Building query based on filter {{(pid=124013) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3878}}
Jul 02 20:50:58.812724 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_store_chunk_size      = 4 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.813087 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.rbd_user                  = cinder {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.813499 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.replication_connect_timeout = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.813901 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.replication_device        = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.814470 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.report_discard_supported  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.814878 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.report_dynamic_total_capacity = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.815277 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.reserved_percentage       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.815676 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.scst_target_driver        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.816143 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.scst_target_iqn_name      = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.816494 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.storage_protocol          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.816936 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.suppress_requests_ssl_warnings = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.817386 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.target_helper             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.817913 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.target_ip_address         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.818254 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.target_port               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.818684 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.target_prefix             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.819270 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.target_protocol           = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.819706 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.trace_flags               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.820167 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.unique_fqdn_network       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.820604 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.use_chap_auth             = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.821039 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.use_multipath_for_image_xfer = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.821577 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_backend_name       = ceph {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.822141 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_clear              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.822705 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_clear_ionice       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.823272 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_clear_size         = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.823821 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_copy_blkio_cgroup_name = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.824409 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_copy_bps_limit     = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.824970 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_dd_blocksize       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.825534 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volume_driver             = cinder.volume.drivers.rbd.RBDDriver {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.826092 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ceph.volumes_dir               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.826737 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.amqp_auto_delete = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.827353 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.amqp_durable_queues = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.830562 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.conn_pool_min_size = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.831184 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.conn_pool_ttl = 1200 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.831804 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.direct_mandatory_flag = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.832589 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.enable_cancel_on_failover = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.833297 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.heartbeat_in_pthread = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.833856 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.heartbeat_rate = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.834388 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.image_utils [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Configuration image_conversion_dir is None or the path doesn't exist. {{(pid=124013) cleanup_temporary_file /opt/stack/cinder/cinder/image/image_utils.py:991}}
Jul 02 20:50:58.835152 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.heartbeat_timeout_threshold = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.835728 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.kombu_compression = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.836350 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.kombu_failover_strategy = round-robin {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.837291 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.kombu_missing_consumer_retry_timeout = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.838244 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.kombu_reconnect_delay = 1.0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.839132 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_ha_queues = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.840091 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_interval_max = 30 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.841093 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_login_method = AMQPLAIN {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.841685 ncdv-master-1 cinder-volume[124013]: INFO cinder.keymgr.migration [None req-f1850fb0-d0a2-45a7-a85a-b74460414f50 None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Jul 02 20:50:58.842364 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_qos_prefetch_count = 0 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.843159 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_quorum_queue = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.843755 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_retry_backoff = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.844348 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_retry_interval = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.845338 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rabbit_transient_queues_ttl = 1800 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.846884 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.rpc_conn_pool_size = 30 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.847638 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.ssl      = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.848588 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.ssl_ca_file =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.849207 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.ssl_cert_file =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.849777 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.ssl_enforce_fips_mode = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.850485 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.ssl_key_file =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.851732 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_rabbit.ssl_version =  {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.852831 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_notifications.driver = [] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.853524 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:58.853884 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_notifications.retry = -1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.854747 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_notifications.topics = ['notifications'] {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.855674 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] oslo_messaging_notifications.transport_url = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.856566 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] key_manager.backend            = cinder.keymgr.conf_key_mgr.ConfKeyManager {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.857240 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] key_manager.fixed_key          = **** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.857882 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.auth_endpoint         = http://localhost/identity/v3 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.858456 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.barbican_api_version  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.859021 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.barbican_endpoint     = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.859608 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.barbican_endpoint_type = public {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.860206 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.barbican_region_name  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.861232 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.cafile                = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.861825 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.certfile              = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.862450 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.collect_timing        = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.863003 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.insecure              = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.863550 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.keyfile               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.864186 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.number_of_retries     = 60 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.864755 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.retry_delay           = 1 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.865402 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.send_service_user_token = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.865852 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.split_loggers         = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.866294 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.timeout               = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.866769 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.verify_ssl            = True {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.867212 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican.verify_ssl_path       = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.867676 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.auth_section = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.868291 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.auth_type = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.869057 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.cafile   = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.869960 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.certfile = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.870513 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.collect_timing = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.871308 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.insecure = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.871817 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.keyfile  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.872256 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.split_loggers = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.872663 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] barbican_service_user.timeout  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.873147 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.approle_role_id          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.873625 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.approle_secret_id        = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.874024 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.cafile                   = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.874632 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.certfile                 = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.875088 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.collect_timing           = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.875586 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.insecure                 = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.876152 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.keyfile                  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.876733 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.kv_mountpoint            = secret {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.877146 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.kv_version               = 2 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.877560 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.namespace                = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.877966 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.root_token_id            = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.878433 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.split_loggers            = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.878842 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.ssl_ca_crt_file          = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.879242 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.timeout                  = None {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.879653 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.use_ssl                  = False {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.880123 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] vault.vault_url                = http://127.0.0.1:8200 {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}}
Jul 02 20:50:58.880452 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] ******************************************************************************** {{(pid=123840) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}}
Jul 02 20:50:58.899439 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:58.929383 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:50:58.953897 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Driver initialization completed successfully.
Jul 02 20:50:58.954656 ncdv-master-1 cinder-volume[124013]: INFO cinder.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Initiating service 3 cleanup
Jul 02 20:50:58.964274 ncdv-master-1 cinder-volume[124013]: INFO cinder.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Service 3 cleanup completed.
Jul 02 20:50:58.964940 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.service [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Creating RPC server for service cinder-volume {{(pid=124013) start /opt/stack/cinder/cinder/service.py:231}}
Jul 02 20:50:58.972838 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.service [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=124013) start /opt/stack/cinder/cinder/service.py:237}}
Jul 02 20:50:59.021004 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
Jul 02 20:50:59.022402 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:59.050205 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:50:59.110954 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1c5aca57-cf97-4a27-99f8-ad13688b3253 None None] Driver post RPC initialization completed successfully.
Jul 02 20:51:08.549639 ncdv-master-1 cinder-volume[123840]: DEBUG dbcounter [-] [123840] Writing DB stats cinder:SELECT=16,cinder:INSERT=1 {{(pid=123840) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:51:09.098601 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=22,cinder:INSERT=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:51:09.107471 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:51:29.113828 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:51:39.127647 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:51:59.126407 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:52:09.129648 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:52:34.114921 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:52:34.115735 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:52:34.140082 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:52:34.163475 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:52:39.123868 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:52:49.121913 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:53:09.124660 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:53:29.118089 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:53:34.119026 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:53:34.119676 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:53:34.144741 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:53:34.175655 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:53:49.130060 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:53:49.131593 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:54:19.139579 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:54:19.144976 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:54:34.121418 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:54:34.122350 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:54:34.152572 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:54:34.180642 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:54:39.143005 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:54:59.133931 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:UPDATE=2,cinder:SELECT=8 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:55:19.131381 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:55:19.135655 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:55:34.126275 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:55:34.128367 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:55:34.154753 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:55:34.181431 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:55:59.131235 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:55:59.133481 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:56:34.126984 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:56:34.127648 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:56:34.150866 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:56:34.174537 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:56:39.133556 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:56:39.135557 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:57:09.138904 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:57:09.148643 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:57:34.131797 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:57:34.132513 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:57:34.155556 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:57:34.178492 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:57:39.139473 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:57:49.138681 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:58:09.134547 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:58:09.148434 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:58:34.133569 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:58:34.134228 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:58:34.156738 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:58:34.179723 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:58:49.122984 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:58:49.124636 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:59:19.133655 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:59:19.134703 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:59:29.151039 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:59:34.134532 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 20:59:34.135190 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:59:34.158267 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 20:59:34.181316 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 20:59:59.148383 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 20:59:59.150221 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:00:29.149260 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:00:34.137210 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:00:34.137983 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:00:34.164091 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:00:34.187806 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:00:39.144095 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:00:59.153821 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:01:09.155686 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:01:29.166031 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=4,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:01:34.133070 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:01:34.133694 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:01:34.204957 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:01:34.262368 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:01:49.150966 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:02:09.162487 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:02:29.165436 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:02:34.137942 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:02:34.138594 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:02:34.205178 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:02:34.318780 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:02:39.177668 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:02:59.182475 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=6,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:03:09.194385 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:03:29.179434 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:03:32.195718 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:03:32.318690 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (3bd125fd-61de-480b-95f6-a3c209208ccd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:03:32.340182 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (caf17e28-43ce-4e61-ba1c-677398b1b16b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:03:32.493925 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (caf17e28-43ce-4e61-ba1c-677398b1b16b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:03:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-426527257',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=0fd88419-3147-47fa-bed8-27c867972435,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='63b8773841e44baab20aa0677e027de7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:03:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:03:32Z,use_quota=True,user_id='45991293f65c4c22a91afa74cf8179ce',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:03:32.514959 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c8881fea-28e4-42b4-a12d-bf516d9a58d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:03:32.522718 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c8881fea-28e4-42b4-a12d-bf516d9a58d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:03:32.537152 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7f0b0595-1c31-4588-9d02-52fab958760b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:03:32.690572 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (7f0b0595-1c31-4588-9d02-52fab958760b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '0fd88419-3147-47fa-bed8-27c867972435', 'volume_name': 'volume-0fd88419-3147-47fa-bed8-27c867972435', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807b5c64f0>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:03:32.695170 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (22f0f911-cfe4-46f3-b949-fcbc8dcfdc97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:03:32.708833 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (22f0f911-cfe4-46f3-b949-fcbc8dcfdc97) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:03:32.725464 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d69b9997-2ebf-4d31-912a-184154528765) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:03:32.726604 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume 0fd88419-3147-47fa-bed8-27c867972435: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-0fd88419-3147-47fa-bed8-27c867972435', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807b5c64f0>}
Jul 02 21:03:32.727196 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Cloning 0fd88419-3147-47fa-bed8-27c867972435 from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=124013) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:03:32.728093 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:03:32.757385 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.029s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:03:32.784681 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Preparing cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:889}}
Jul 02 21:03:32.788168 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:03:34.134913 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:03:34.138575 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:03:34.210807 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:03:34.299709 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:03:49.177542 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=11,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:03:52.255191 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Image fetch details: dest /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph, size 616.00 MB, duration 19.44 sec {{(pid=124013) fetch /opt/stack/cinder/cinder/image/image_utils.py:587}}
Jul 02 21:03:52.256033 ncdv-master-1 cinder-volume[124013]: INFO cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Image download 616.00 MB at 31.68 MB/s
Jul 02 21:03:52.316610 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:03:52.507047 ncdv-master-1 sudo[136272]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph
Jul 02 21:03:52.508267 ncdv-master-1 sudo[136272]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jul 02 21:03:54.115978 ncdv-master-1 sudo[136272]: pam_unix(sudo:session): session closed for user root
Jul 02 21:03:54.118171 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph" returned: 0 in 1.801s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:03:54.123886 ncdv-master-1 cinder-volume[124013]: WARNING py.warnings [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena'
Jul 02 21:03:54.123886 ncdv-master-1 cinder-volume[124013]:   debtcollector.deprecate(
Jul 02 21:03:54.123886 ncdv-master-1 cinder-volume[124013]: 
Jul 02 21:03:54.126091 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Temporary image e482b755-1c1a-4758-a7c0-b3edf1610541 is fetched for user 45991293f65c4c22a91afa74cf8179ce. {{(pid=124013) fetch /opt/stack/cinder/cinder/image/image_utils.py:1124}}
Jul 02 21:03:54.221369 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:03:54.334682 ncdv-master-1 sudo[136304]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph
Jul 02 21:03:54.335531 ncdv-master-1 sudo[136304]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jul 02 21:03:55.561117 ncdv-master-1 sudo[136304]: pam_unix(sudo:session): session closed for user root
Jul 02 21:03:55.563299 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph" returned: 0 in 1.342s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:03:55.564984 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] creating volume 'volume-0fd88419-3147-47fa-bed8-27c867972435' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:03:55.575255 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:03:55.774058 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:03:55.774439 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:03:55.804811 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.volume_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Attempting download of e482b755-1c1a-4758-a7c0-b3edf1610541 ((None, None)) to volume 0fd88419-3147-47fa-bed8-27c867972435. {{(pid=124013) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1181}}
Jul 02 21:03:55.882467 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_e482b755-1c1a-4758-a7c0-b3edf1610541_72limczd {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:03:55.979963 ncdv-master-1 sudo[136330]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_e482b755-1c1a-4758-a7c0-b3edf1610541_72limczd
Jul 02 21:03:55.980836 ncdv-master-1 sudo[136330]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jul 02 21:03:57.218009 ncdv-master-1 sudo[136330]: pam_unix(sudo:session): session closed for user root
Jul 02 21:03:57.221181 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_download_e482b755-1c1a-4758-a7c0-b3edf1610541_72limczd" returned: 0 in 1.338s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:03:57.223449 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:03:57.325392 ncdv-master-1 sudo[136335]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph
Jul 02 21:03:57.326213 ncdv-master-1 sudo[136335]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jul 02 21:03:58.634205 ncdv-master-1 sudo[136335]: pam_unix(sudo:session): session closed for user root
Jul 02 21:03:58.637178 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph" returned: 0 in 1.413s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:03:58.638582 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] e482b755-1c1a-4758-a7c0-b3edf1610541 was vhdx, converting to raw {{(pid=124013) fetch_to_volume_format /opt/stack/cinder/cinder/image/image_utils.py:825}}
Jul 02 21:03:58.640474 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpf5m9q4w8 {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:03:58.661750 ncdv-master-1 sudo[136350]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpf5m9q4w8
Jul 02 21:03:58.662623 ncdv-master-1 sudo[136350]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jul 02 21:04:01.109693 ncdv-master-1 sudo[136350]: pam_unix(sudo:session): session closed for user root
Jul 02 21:04:01.111892 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "sudo cinder-rootwrap /etc/cinder/rootwrap.conf qemu-img convert -O raw -f vhdx /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph /opt/stack/data/cinder/conversion/tmpf5m9q4w8" returned: 0 in 2.471s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:01.113547 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): /usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:01.203156 ncdv-master-1 sudo[136383]:   ubuntu : TTY=unknown ; PWD=/ ; USER=root ; COMMAND=/usr/local/bin/cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph
Jul 02 21:04:01.204524 ncdv-master-1 sudo[136383]: pam_unix(sudo:session): session opened for user root by (uid=0)
Jul 02 21:04:02.622646 ncdv-master-1 sudo[136383]: pam_unix(sudo:session): session closed for user root
Jul 02 21:04:02.624963 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "/usr/bin/python3.8 -m oslo_concurrency.prlimit --as=1073741824 --cpu=60 -- sudo cinder-rootwrap /etc/cinder/rootwrap.conf env LC_ALL=C qemu-img info /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph" returned: 0 in 1.511s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:02.625697 ncdv-master-1 cinder-volume[124013]: WARNING py.warnings [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] /usr/local/lib/python3.8/dist-packages/oslo_utils/imageutils.py:71: FutureWarning: The human format is deprecated and the format parameter will be removed. Use explicitly json instead in version 'xena'
Jul 02 21:04:02.625697 ncdv-master-1 cinder-volume[124013]:   debtcollector.deprecate(
Jul 02 21:04:02.625697 ncdv-master-1 cinder-volume[124013]: 
Jul 02 21:04:02.626340 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Image conversion details: src /opt/stack/data/cinder/conversion/image_fetch_e482b755-1c1a-4758-a7c0-b3edf1610541_xuupa3kyncdv-master-1@ceph, size 700.00 MB, duration 2.47 sec, destination /opt/stack/data/cinder/conversion/tmpf5m9q4w8 {{(pid=124013) _convert_image /opt/stack/cinder/cinder/image/image_utils.py:419}}
Jul 02 21:04:02.626998 ncdv-master-1 cinder-volume[124013]: INFO cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Converted 700.00 MB image at 283.02 MB/s
Jul 02 21:04:02.628482 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:02.717951 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:04:02.720276 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:04:02.748470 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] deleting rbd volume volume-0fd88419-3147-47fa-bed8-27c867972435 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:04:03.107678 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpf5m9q4w8 volume-0fd88419-3147-47fa-bed8-27c867972435 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:08.101594 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "rbd import --pool volumes --order 22 /opt/stack/data/cinder/conversion/tmpf5m9q4w8 volume-0fd88419-3147-47fa-bed8-27c867972435 --new-format --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 4.994s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:08.242476 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:08.372442 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:04:08.372920 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:04:08.373445 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.volume_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Downloaded image e482b755-1c1a-4758-a7c0-b3edf1610541 ((None, None)) to volume 0fd88419-3147-47fa-bed8-27c867972435 successfully. {{(pid=124013) copy_image_to_volume /opt/stack/cinder/cinder/volume/volume_utils.py:1223}}
Jul 02 21:04:08.440439 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.image_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Temporary image e482b755-1c1a-4758-a7c0-b3edf1610541 for user 45991293f65c4c22a91afa74cf8179ce is deleted. {{(pid=124013) fetch /opt/stack/cinder/cinder/image/image_utils.py:1128}}
Jul 02 21:04:08.477390 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=13,cinder:UPDATE=4,cinder:INSERT=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:04:08.500091 ncdv-master-1 cinder-volume[124013]: WARNING cinder.quota [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:04:08.500888 ncdv-master-1 cinder-volume[124013]: WARNING cinder.quota [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:04:08.577915 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Created reservations ['adf8683d-6305-4de9-a7f5-1239acfa48eb', 'd1139993-ed68-4fa0-a0b0-f414084dd9cd', '93cd5bd7-1eb0-483d-8268-05e8a7a1e34b', '28f4459c-2169-49d9-a47f-b6b87ff212da'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:04:08.594891 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Creating image volume entry: {'group_id': None, 'multiattach': False, 'volume_metadata': [], 'size': 1, 'updated_at': datetime.datetime(2022, 7, 2, 21, 3, 32, tzinfo=datetime.timezone.utc), 'provider_location': None, 'service_uuid': None, 'volume_glance_metadata': [{'key': 'signature_verified', 'value': 'False'}], 'display_description': None, 'metadata': {}, 'cluster_name': None, 'consistencygroup_id': None, 'deleted': False, 'project_id': '001bf63308784f4eb62e64d6f82c8354', 'shared_targets': True, 'provider_auth': None, 'launched_at': None, 'replication_status': None, 'host': 'ncdv-master-1@ceph#ceph', 'bootable': False, 'replication_driver_data': None, 'scheduled_at': datetime.datetime(2022, 7, 2, 21, 3, 32, tzinfo=datetime.timezone.utc), 'display_name': 'image-e482b755-1c1a-4758-a7c0-b3edf1610541', 'user_id': '45991293f65c4c22a91afa74cf8179ce', 'availability_zone': 'nova', 'provider_id': None, 'deleted_at': None, 'volume_admin_metadata': [], 'ec2_id': None, 'previous_status': None, 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'created_at': datetime.datetime(2022, 7, 2, 21, 3, 31, tzinfo=datetime.timezone.utc), 'terminated_at': None, 'provider_geometry': None, 'encryption_key_id': None, 'replication_extended_status': None, 'attach_status': 'detached', 'status': 'creating', 'source_volid': '0fd88419-3147-47fa-bed8-27c867972435'}. {{(pid=124013) _clone_image_volume /opt/stack/cinder/cinder/volume/manager.py:1620}}
Jul 02 21:04:08.696792 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume reschedule parameters: False retry: None {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:04:08.722192 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (c9d10808-eddc-4534-bd95-4af767d07c1b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:08.728748 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9accda09-c6c5-4c2e-868f-12febaf828e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:08.747819 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9accda09-c6c5-4c2e-868f-12febaf828e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata={},attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:03:31Z,deleted=False,deleted_at=None,display_description=None,display_name='image-e482b755-1c1a-4758-a7c0-b3edf1610541',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=12fc5816-9c32-4ebe-a0f8-30a641fe4f45,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='001bf63308784f4eb62e64d6f82c8354',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:03:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=0fd88419-3147-47fa-bed8-27c867972435,status='creating',terminated_at=None,updated_at=2022-07-02T21:03:32Z,use_quota=True,user_id='45991293f65c4c22a91afa74cf8179ce',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:08.753501 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (adb8f676-0e9d-4654-a926-cb4f37a7565e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:08.757424 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (adb8f676-0e9d-4654-a926-cb4f37a7565e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:08.762205 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (67fde531-62ff-4bb6-93a5-d418b4fa7e32) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:08.778584 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (67fde531-62ff-4bb6-93a5-d418b4fa7e32) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'source_vol', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'volume_name': 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'volume_size': 1, 'source_volid': '0fd88419-3147-47fa-bed8-27c867972435', 'source_volstatus': 'downloading'}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:08.783841 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6cb4abd2-b8c8-4568-a8e1-da1f1264b051) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:08.787700 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (6cb4abd2-b8c8-4568-a8e1-da1f1264b051) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:08.792162 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (726575ad-4fb5-4e79-bca0-47b30ceaaf4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:08.793224 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45: being created as source_vol with specification: {'status': 'creating', 'volume_name': 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'volume_size': 1, 'source_volid': '0fd88419-3147-47fa-bed8-27c867972435', 'source_volstatus': 'downloading'}
Jul 02 21:04:08.805740 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:08.893751 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] creating snapshot='volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45.clone_snap' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:04:09.055387 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] cloning 'volume-0fd88419-3147-47fa-bed8-27c867972435@volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45.clone_snap' to 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}}
Jul 02 21:04:09.216635 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:04:09.217989 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:04:09.218491 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:04:09.255564 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] clone created successfully {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}}
Jul 02 21:04:09.260661 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (726575ad-4fb5-4e79-bca0-47b30ceaaf4c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'volume_size': 1, 'source_volid': '0fd88419-3147-47fa-bed8-27c867972435', 'source_volstatus': 'downloading'}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:09.266788 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cea2a92f-e96e-4a9a-a2c1-576ac7b01b45) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:09.279251 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45 (12fc5816-9c32-4ebe-a0f8-30a641fe4f45): created successfully
Jul 02 21:04:09.284496 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cea2a92f-e96e-4a9a-a2c1-576ac7b01b45) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:09.291348 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (c9d10808-eddc-4534-bd95-4af767d07c1b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:09.303671 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Created volume successfully.
Jul 02 21:04:09.371825 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Creating new image-volume cache entry for image e482b755-1c1a-4758-a7c0-b3edf1610541 on ncdv-master-1@ceph#ceph {{(pid=124013) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:104}}
Jul 02 21:04:09.381881 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] New image-volume cache entry created: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 4, 9, 375187)}. {{(pid=124013) create_cache_entry /opt/stack/cinder/cinder/image/cache.py:129}}
Jul 02 21:04:09.390347 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 36.633s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:04:09.391285 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.volume_utils [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Marking volume 0fd88419-3147-47fa-bed8-27c867972435 as bootable. {{(pid=124013) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}}
Jul 02 21:04:09.402369 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Copying metadata from image e482b755-1c1a-4758-a7c0-b3edf1610541 to 0fd88419-3147-47fa-bed8-27c867972435. {{(pid=124013) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}}
Jul 02 21:04:09.402890 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Creating volume glance metadata for volume 0fd88419-3147-47fa-bed8-27c867972435 backed by image e482b755-1c1a-4758-a7c0-b3edf1610541 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=124013) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}}
Jul 02 21:04:09.483222 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d69b9997-2ebf-4d31-912a-184154528765) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-0fd88419-3147-47fa-bed8-27c867972435', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807b5c64f0>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:09.489726 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (20141797-a509-4a7d-8c00-b7bafab7ab7b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:09.505573 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435 (0fd88419-3147-47fa-bed8-27c867972435): created successfully
Jul 02 21:04:09.512100 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (20141797-a509-4a7d-8c00-b7bafab7ab7b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:09.519757 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (3bd125fd-61de-480b-95f6-a3c209208ccd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:09.532904 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-b2f03fde-9952-4be2-a19e-95489430783d tempest-TaggedBootDevicesTest_v242-1793629843 None] Created volume successfully.
Jul 02 21:04:10.157446 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:04:10.193352 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (fb679c8e-1c3b-4670-b4b9-297c48a08416) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:10.200566 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f497f15b-2236-4bce-a4b1-6e0279f2d2b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:10.219441 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (f497f15b-2236-4bce-a4b1-6e0279f2d2b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:04:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1505631664',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=6da0916e-d49c-4e14-94a5-3cccd3b7c22c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='63b8773841e44baab20aa0677e027de7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:04:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:04:10Z,use_quota=True,user_id='45991293f65c4c22a91afa74cf8179ce',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:10.233762 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ca1a7ea-f68b-469e-ac8f-7ed26d97b172) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:10.245867 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (8ca1a7ea-f68b-469e-ac8f-7ed26d97b172) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:10.257057 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c1f9b7f-2b49-4ad3-82cf-8a645ab882e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:10.268402 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (5c1f9b7f-2b49-4ad3-82cf-8a645ab882e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'volume_name': 'volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:10.273774 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa989108-43d0-45f6-8176-e4d98cdafad0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:10.277944 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fa989108-43d0-45f6-8176-e4d98cdafad0) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:10.282410 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (07e90217-b329-4d67-97b7-61a76ddc0a1c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:10.283468 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume 6da0916e-d49c-4e14-94a5-3cccd3b7c22c: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'volume_size': 1}
Jul 02 21:04:10.283866 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] creating volume 'volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:04:10.284999 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:10.362093 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:04:10.362728 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:04:10.368431 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (07e90217-b329-4d67-97b7-61a76ddc0a1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:10.372666 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (981bd0c9-a1fc-4476-9f34-6128227f08f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:10.387232 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c (6da0916e-d49c-4e14-94a5-3cccd3b7c22c): created successfully
Jul 02 21:04:10.393231 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (981bd0c9-a1fc-4476-9f34-6128227f08f7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:10.401934 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (fb679c8e-1c3b-4670-b4b9-297c48a08416) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:10.415196 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-d89de54d-330e-41b5-a254-98a8ceefcc19 tempest-TaggedBootDevicesTest_v242-1793629843 None] Created volume successfully.
Jul 02 21:04:11.916736 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:04:11.960484 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (20690ea2-3adb-496c-a5d9-e77efdc4f9a1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:11.967165 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0d095b0c-27e1-4107-9514-fe86c7b977ca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:11.985022 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0d095b0c-27e1-4107-9514-fe86c7b977ca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:04:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest_v242-volume-1394271922',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=4af561d5-4d61-49c9-b2df-3812b0172ae2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='63b8773841e44baab20aa0677e027de7',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:04:12Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:04:12Z,use_quota=True,user_id='45991293f65c4c22a91afa74cf8179ce',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:11.990558 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ba11f0af-3a4f-4e6d-9803-bf3b74656ab4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:11.995015 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ba11f0af-3a4f-4e6d-9803-bf3b74656ab4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:12.000120 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f09af14c-a41b-49e2-b767-ee524365bc3d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:12.005806 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f09af14c-a41b-49e2-b767-ee524365bc3d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4af561d5-4d61-49c9-b2df-3812b0172ae2', 'volume_name': 'volume-4af561d5-4d61-49c9-b2df-3812b0172ae2', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:12.018816 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d7cc9326-be0c-40bf-aab3-6122b1fcc4b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:12.026692 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d7cc9326-be0c-40bf-aab3-6122b1fcc4b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:12.059413 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f396752b-a0e0-4574-9acc-a8882bc72226) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:12.067318 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume 4af561d5-4d61-49c9-b2df-3812b0172ae2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4af561d5-4d61-49c9-b2df-3812b0172ae2', 'volume_size': 1}
Jul 02 21:04:12.067821 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] creating volume 'volume-4af561d5-4d61-49c9-b2df-3812b0172ae2' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:04:12.069563 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:12.177707 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:04:12.178222 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:04:12.182073 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (f396752b-a0e0-4574-9acc-a8882bc72226) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4af561d5-4d61-49c9-b2df-3812b0172ae2', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:12.186410 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2d24653e-c86c-4ba6-98c9-046a4dd55dfe) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:12.196910 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-4af561d5-4d61-49c9-b2df-3812b0172ae2 (4af561d5-4d61-49c9-b2df-3812b0172ae2): created successfully
Jul 02 21:04:12.201219 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2d24653e-c86c-4ba6-98c9-046a4dd55dfe) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:12.208168 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Flow 'volume_create_manager' (20690ea2-3adb-496c-a5d9-e77efdc4f9a1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:12.217315 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-3aa901ae-fc9b-491c-87b2-229c8652183e tempest-TaggedBootDevicesTest_v242-1793629843 None] Created volume successfully.
Jul 02 21:04:29.181384 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=64,cinder:UPDATE=16,cinder:INSERT=17,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:04:34.136422 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:04:34.137309 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:34.209213 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:34.283326 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:04:37.886739 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fb3c014b-c143-4211-bc8b-698421b60a9f tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:39.207074 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=36,cinder:INSERT=7,cinder:UPDATE=13,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:04:39.545062 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:04:39.606587 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Flow 'volume_create_manager' (8fca311c-b1b2-4978-a218-e1b2bb117e12) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:39.634888 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (be1f45a5-4d1c-48f8-8266-86677baac61e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:39.678888 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (be1f45a5-4d1c-48f8-8266-86677baac61e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:04:39Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveAutoBlockMigrationV225Test-volume-3104666',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=48d74eca-fcde-4742-9dbc-87d3e9f0e8c2,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d59cb1f0d7bb4708ac0994f7c9b846b3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:04:39Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:04:39Z,use_quota=True,user_id='f3287fb85c2343d686718d3c4aaa5c00',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:39.684416 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cea973df-3ca9-4234-857f-7fbfe470bf3e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:39.689736 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (cea973df-3ca9-4234-857f-7fbfe470bf3e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:39.695272 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bd3a76db-abfa-4d50-9e3a-a8eaefa18899) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:39.702339 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bd3a76db-abfa-4d50-9e3a-a8eaefa18899) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'volume_name': 'volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:39.707700 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (406e3bed-36e7-4941-b11f-79b3a334b7b9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:39.718347 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (406e3bed-36e7-4941-b11f-79b3a334b7b9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:39.724658 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b10e54bb-8954-4691-a920-68e7216144fb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:39.726006 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume 48d74eca-fcde-4742-9dbc-87d3e9f0e8c2: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'volume_size': 1}
Jul 02 21:04:39.727101 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] creating volume 'volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:04:39.728076 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:04:39.780491 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fb3c014b-c143-4211-bc8b-698421b60a9f tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.894s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:39.782221 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fb3c014b-c143-4211-bc8b-698421b60a9f tempest-TaggedBootDevicesTest_v242-1793629843 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-0fd88419-3147-47fa-bed8-27c867972435', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '0fd88419-3147-47fa-bed8-27c867972435', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:04:39.857373 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fb3c014b-c143-4211-bc8b-698421b60a9f tempest-TaggedBootDevicesTest_v242-1793629843 None] Connection info returned from driver {'name': 'volumes/volume-0fd88419-3147-47fa-bed8-27c867972435', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '0fd88419-3147-47fa-bed8-27c867972435', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '6a1029c5-e65f-48cf-bb48-a696e1214fe1'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:04:39.904324 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:04:39.905015 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:04:39.909190 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b10e54bb-8954-4691-a920-68e7216144fb) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:39.920320 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b3c24b41-9b9f-4101-9d2e-99bf48b99fae) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:04:39.951309 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2 (48d74eca-fcde-4742-9dbc-87d3e9f0e8c2): created successfully
Jul 02 21:04:39.955174 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b3c24b41-9b9f-4101-9d2e-99bf48b99fae) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:04:39.961591 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Flow 'volume_create_manager' (8fca311c-b1b2-4978-a218-e1b2bb117e12) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:04:39.994248 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-4ad2ede6-ae92-499f-b9ae-e324c15de8a9 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Created volume successfully.
Jul 02 21:04:40.038018 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fb3c014b-c143-4211-bc8b-698421b60a9f tempest-TaggedBootDevicesTest_v242-1793629843 None] attachment_update completed successfully.
Jul 02 21:04:41.241017 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-e09d84d0-763c-484c-9dc1-ea260c5d6120 tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:42.786957 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-e09d84d0-763c-484c-9dc1-ea260c5d6120 tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.546s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:42.790151 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-e09d84d0-763c-484c-9dc1-ea260c5d6120 tempest-TaggedBootDevicesTest_v242-1793629843 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:04:42.877592 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-e09d84d0-763c-484c-9dc1-ea260c5d6120 tempest-TaggedBootDevicesTest_v242-1793629843 None] Connection info returned from driver {'name': 'volumes/volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6da0916e-d49c-4e14-94a5-3cccd3b7c22c', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a34492cc-0c39-4d38-be6b-702e55f3a576'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:04:43.005579 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-e09d84d0-763c-484c-9dc1-ea260c5d6120 tempest-TaggedBootDevicesTest_v242-1793629843 None] attachment_update completed successfully.
Jul 02 21:04:43.177487 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-4bd97a87-f2ee-4c50-9ef1-278e75edaafe req-888ca1bf-aee9-4e55-b24d-9420d942aef5 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:44.376681 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-4bd97a87-f2ee-4c50-9ef1-278e75edaafe req-888ca1bf-aee9-4e55-b24d-9420d942aef5 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.199s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:44.378416 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bd97a87-f2ee-4c50-9ef1-278e75edaafe req-888ca1bf-aee9-4e55-b24d-9420d942aef5 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:04:44.434202 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-4bd97a87-f2ee-4c50-9ef1-278e75edaafe req-888ca1bf-aee9-4e55-b24d-9420d942aef5 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Connection info returned from driver {'name': 'volumes/volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '746f9438-1475-4c8d-b45d-e33701b54cf2'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:04:44.526545 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-4bd97a87-f2ee-4c50-9ef1-278e75edaafe req-888ca1bf-aee9-4e55-b24d-9420d942aef5 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] attachment_update completed successfully.
Jul 02 21:04:46.384729 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fe9a7154-e4c3-48d8-9efd-f457880e32c4 tempest-TaggedBootDevicesTest_v242-1793629843 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:47.647979 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fe9a7154-e4c3-48d8-9efd-f457880e32c4 tempest-TaggedBootDevicesTest_v242-1793629843 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.265s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:47.651152 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fe9a7154-e4c3-48d8-9efd-f457880e32c4 tempest-TaggedBootDevicesTest_v242-1793629843 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4af561d5-4d61-49c9-b2df-3812b0172ae2', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '4af561d5-4d61-49c9-b2df-3812b0172ae2', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:04:47.684689 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fe9a7154-e4c3-48d8-9efd-f457880e32c4 tempest-TaggedBootDevicesTest_v242-1793629843 None] Connection info returned from driver {'name': 'volumes/volume-4af561d5-4d61-49c9-b2df-3812b0172ae2', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '4af561d5-4d61-49c9-b2df-3812b0172ae2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '9fd01942-b0c2-401e-be58-0716ed6493ee'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:04:47.822064 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-918a3b6e-874a-4bd5-b9c6-43c50860f4e7 req-fe9a7154-e4c3-48d8-9efd-f457880e32c4 tempest-TaggedBootDevicesTest_v242-1793629843 None] attachment_update completed successfully.
Jul 02 21:04:58.045766 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-9674a725-c98f-4f8e-b771-8252f725e6db tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:04:59.571816 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-9674a725-c98f-4f8e-b771-8252f725e6db tempest-LiveAutoBlockMigrationV225Test-2025202973 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.529s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:04:59.573168 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-9674a725-c98f-4f8e-b771-8252f725e6db tempest-LiveAutoBlockMigrationV225Test-2025202973 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:04:59.580800 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=55,cinder:UPDATE=8 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:04:59.681213 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-9674a725-c98f-4f8e-b771-8252f725e6db tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Connection info returned from driver {'name': 'volumes/volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '48d74eca-fcde-4742-9dbc-87d3e9f0e8c2', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '95d7eca3-ae53-4232-bd59-d34a10cbcedd'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:04:59.788753 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-9674a725-c98f-4f8e-b771-8252f725e6db tempest-LiveAutoBlockMigrationV225Test-2025202973 None] attachment_update completed successfully.
Jul 02 21:05:12.593771 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=59,cinder:UPDATE=12,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:05:12.608374 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:05:12.628943 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Flow 'volume_create_manager' (bf1eaded-9693-43c4-b7ba-e70575f1c014) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:05:12.635437 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1a87b8f-861e-4bd7-86e6-2ae21aeec26b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:12.654327 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e1a87b8f-861e-4bd7-86e6-2ae21aeec26b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:05:12Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-DeleteServersTestJSON-volume-393640573',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=c2aa26fe-49ef-4733-8bdc-f5d9113ce416,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8576652bfe8441e0985297d016a1ca9b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:05:13Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:05:13Z,use_quota=True,user_id='3fefe83ec0664f6188af74ea4b5c1291',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:12.659402 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (acebda49-6b1d-4005-9087-b1a45a08d480) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:12.663293 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (acebda49-6b1d-4005-9087-b1a45a08d480) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:12.668025 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (47c314fc-b283-46c1-9fd5-6ebcf7e9745c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:12.671953 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (47c314fc-b283-46c1-9fd5-6ebcf7e9745c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'volume_name': 'volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:12.676850 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a18ba6e6-2e1a-486b-923e-bd8ba3898221) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:12.680615 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (a18ba6e6-2e1a-486b-923e-bd8ba3898221) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:12.685033 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (218dd9f7-c646-476d-96a9-a3a811bb0be3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:12.686129 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Volume c2aa26fe-49ef-4733-8bdc-f5d9113ce416: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'volume_size': 1}
Jul 02 21:05:12.686549 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] creating volume 'volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:05:12.687310 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:05:12.746582 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:05:12.747024 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:05:12.750720 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (218dd9f7-c646-476d-96a9-a3a811bb0be3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:12.755017 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07ab8161-d352-4050-99ad-6a0283afa217) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:12.765600 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Volume volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416 (c2aa26fe-49ef-4733-8bdc-f5d9113ce416): created successfully
Jul 02 21:05:12.769491 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (07ab8161-d352-4050-99ad-6a0283afa217) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:12.775010 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Flow 'volume_create_manager' (bf1eaded-9693-43c4-b7ba-e70575f1c014) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:05:12.784149 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-84e55826-10e1-4d7f-96cd-ae393e6d713c tempest-DeleteServersTestJSON-1749805065 None] Created volume successfully.
Jul 02 21:05:14.478108 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-0137a7a4-8454-4d41-9f7c-98b794ec3e2c req-e2af0f94-0700-4dd6-875b-b431ac704392 tempest-DeleteServersTestJSON-1749805065 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:05:15.272621 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-0137a7a4-8454-4d41-9f7c-98b794ec3e2c req-e2af0f94-0700-4dd6-875b-b431ac704392 tempest-DeleteServersTestJSON-1749805065 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 0.794s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:05:15.273458 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-0137a7a4-8454-4d41-9f7c-98b794ec3e2c req-e2af0f94-0700-4dd6-875b-b431ac704392 tempest-DeleteServersTestJSON-1749805065 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:05:15.300963 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-0137a7a4-8454-4d41-9f7c-98b794ec3e2c req-e2af0f94-0700-4dd6-875b-b431ac704392 tempest-DeleteServersTestJSON-1749805065 None] Connection info returned from driver {'name': 'volumes/volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'c2aa26fe-49ef-4733-8bdc-f5d9113ce416', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'bde731c7-5e04-4719-8d20-0562ebc2bb15'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:05:15.365556 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-0137a7a4-8454-4d41-9f7c-98b794ec3e2c req-e2af0f94-0700-4dd6-875b-b431ac704392 tempest-DeleteServersTestJSON-1749805065 None] attachment_update completed successfully.
Jul 02 21:05:19.873595 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-03f768d6-b002-468e-b1f6-37c08814a057 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Terminate volume connection completed successfully.
Jul 02 21:05:19.874374 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-18f3eb5d-c91d-466a-8bf1-0a18a3029b02 req-03f768d6-b002-468e-b1f6-37c08814a057 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Deleting attachment 746f9438-1475-4c8d-b45d-e33701b54cf2. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:05:22.892545 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-03013069-a055-419d-90e4-afcf0f722d7b req-1a6ea496-c259-4184-a04f-c0ea1bd2b013 tempest-DeleteServersTestJSON-1749805065 None] Terminate volume connection completed successfully.
Jul 02 21:05:22.893207 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-03013069-a055-419d-90e4-afcf0f722d7b req-1a6ea496-c259-4184-a04f-c0ea1bd2b013 tempest-DeleteServersTestJSON-1749805065 None] Deleting attachment bde731c7-5e04-4719-8d20-0562ebc2bb15. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:05:34.142101 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:05:34.144019 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:05:34.189985 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:05:34.240858 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:05:34.664070 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=35,cinder:UPDATE=6 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:05:34.674461 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-0e70f7ac-bf1c-41f2-8e65-a276fbe28095 req-8182ae02-9b03-49dc-abb7-24ccd445d66e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Terminate volume connection completed successfully.
Jul 02 21:05:34.674929 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-0e70f7ac-bf1c-41f2-8e65-a276fbe28095 req-8182ae02-9b03-49dc-abb7-24ccd445d66e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Deleting attachment 95d7eca3-ae53-4232-bd59-d34a10cbcedd. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:05:36.071841 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:05:36.123372 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Flow 'volume_create_manager' (3681bc9b-a963-442c-b007-c9c164cbd40a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:05:36.139808 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbe88330-36d7-4655-b19d-bcaa85bf7ddb) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:36.174899 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (cbe88330-36d7-4655-b19d-bcaa85bf7ddb) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:05:36Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-209730363',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=84fa484c-0fe4-4a8b-83a4-e46960aee0a8,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='d59cb1f0d7bb4708ac0994f7c9b846b3',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:05:36Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:05:36Z,use_quota=True,user_id='f3287fb85c2343d686718d3c4aaa5c00',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:36.194844 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb2e1f01-caae-4835-9858-7c2275d5942a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:36.199432 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (eb2e1f01-caae-4835-9858-7c2275d5942a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:36.208928 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (947798da-7999-41fa-89a5-b80ad47ed160) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:36.341005 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (947798da-7999-41fa-89a5-b80ad47ed160) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'volume_name': 'volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072ae1160>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:36.346633 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8141a0e7-9b0a-4ea9-95b3-d51fc20b51ac) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:36.351601 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8141a0e7-9b0a-4ea9-95b3-d51fc20b51ac) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:36.357908 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bbbac85b-d62f-4aa4-8928-a35f64f6da8c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:36.359488 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume 84fa484c-0fe4-4a8b-83a4-e46960aee0a8: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072ae1160>}
Jul 02 21:05:36.360332 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Cloning 84fa484c-0fe4-4a8b-83a4-e46960aee0a8 from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=124013) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:05:36.362245 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:05:36.374210 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.012s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:05:36.394223 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 5, 36, 383184)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:05:36.395244 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:05:36.395850 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Found cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}}
Jul 02 21:05:36.402987 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 0.029s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:05:36.404394 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:05:36.415789 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 5, 36, 409075)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:05:36.417585 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:05:36.418878 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Creating from source image-volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45 {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}}
Jul 02 21:05:36.434035 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:05:36.527308 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] creating snapshot='volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8.clone_snap' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:05:37.612808 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] cloning 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45@volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8.clone_snap' to 'volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}}
Jul 02 21:05:37.807110 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:05:37.809096 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:05:37.810422 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:05:37.853571 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] clone created successfully {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}}
Jul 02 21:05:37.854669 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.volume_utils [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Marking volume 84fa484c-0fe4-4a8b-83a4-e46960aee0a8 as bootable. {{(pid=124013) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}}
Jul 02 21:05:37.872188 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Copying metadata from image e482b755-1c1a-4758-a7c0-b3edf1610541 to 84fa484c-0fe4-4a8b-83a4-e46960aee0a8. {{(pid=124013) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}}
Jul 02 21:05:37.873280 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Creating volume glance metadata for volume 84fa484c-0fe4-4a8b-83a4-e46960aee0a8 backed by image e482b755-1c1a-4758-a7c0-b3edf1610541 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=124013) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}}
Jul 02 21:05:37.994621 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (bbbac85b-d62f-4aa4-8928-a35f64f6da8c) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072ae1160>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:38.006534 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b1277e1-2940-4e13-bb1e-e5def296a2e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:05:38.025927 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8 (84fa484c-0fe4-4a8b-83a4-e46960aee0a8): created successfully
Jul 02 21:05:38.031326 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b1277e1-2940-4e13-bb1e-e5def296a2e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:05:38.039221 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Flow 'volume_create_manager' (3681bc9b-a963-442c-b007-c9c164cbd40a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:05:38.056503 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-72e5946f-1d12-40bb-ab71-b2a75630349d tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Created volume successfully.
Jul 02 21:05:44.070911 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=38,cinder:UPDATE=8,cinder:DELETE=1,cinder:INSERT=7 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:05:44.073953 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-3deca091-df97-4699-8826-7537d7b70ac2 req-c65a699a-51a5-4882-a8c4-65913a296001 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:05:45.635120 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-3deca091-df97-4699-8826-7537d7b70ac2 req-c65a699a-51a5-4882-a8c4-65913a296001 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.561s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:05:45.636496 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-3deca091-df97-4699-8826-7537d7b70ac2 req-c65a699a-51a5-4882-a8c4-65913a296001 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:05:45.692494 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-3deca091-df97-4699-8826-7537d7b70ac2 req-c65a699a-51a5-4882-a8c4-65913a296001 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Connection info returned from driver {'name': 'volumes/volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'a27fbad0-0037-49ed-b574-f52572d491ba'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:05:45.799169 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-3deca091-df97-4699-8826-7537d7b70ac2 req-c65a699a-51a5-4882-a8c4-65913a296001 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] attachment_update completed successfully.
Jul 02 21:05:55.787953 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=29,cinder:UPDATE=6,cinder:INSERT=5,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:06:14.371601 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=18,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:06:14.387231 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-6a080809-797b-4f31-abad-f4fb394c6091 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:06:16.405414 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-6a080809-797b-4f31-abad-f4fb394c6091 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.018s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:06:16.407513 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-6a080809-797b-4f31-abad-f4fb394c6091 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:06:16.462354 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-6a080809-797b-4f31-abad-f4fb394c6091 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Connection info returned from driver {'name': 'volumes/volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '84fa484c-0fe4-4a8b-83a4-e46960aee0a8', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'b525ad5a-723f-4e03-8dfe-969f4d5de83b'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:06:16.626878 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-6a080809-797b-4f31-abad-f4fb394c6091 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] attachment_update completed successfully.
Jul 02 21:06:24.888844 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-615d8ae7-d6b3-474b-b319-193428780a69 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Terminate volume connection completed successfully.
Jul 02 21:06:24.889515 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1eabf886-f8ae-4d97-8da0-e1a848155429 req-615d8ae7-d6b3-474b-b319-193428780a69 tempest-LiveAutoBlockMigrationV225Test-2025202973 None] Deleting attachment a27fbad0-0037-49ed-b574-f52572d491ba. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:06:30.827411 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-f3501544-44aa-432b-bae2-f25d299b2cdd tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Terminate volume connection completed successfully.
Jul 02 21:06:30.827411 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-f3501544-44aa-432b-bae2-f25d299b2cdd tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Deleting attachment b525ad5a-723f-4e03-8dfe-969f4d5de83b. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:06:31.015525 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Acquiring lock "cinder-84fa484c-0fe4-4a8b-83a4-e46960aee0a8-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:06:31.035116 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Lock "cinder-84fa484c-0fe4-4a8b-83a4-e46960aee0a8-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:06:31.076629 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:31.194186 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:06:31.211841 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] deleting rbd volume volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:06:31.610468 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] volume is a clone so cleaning references {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1346}}
Jul 02 21:06:31.648638 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] deleting parent snapshot volume-84fa484c-0fe4-4a8b-83a4-e46960aee0a8.clone_snap {{(pid=124013) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}}
Jul 02 21:06:32.781110 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Acquiring lock "cinder-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:06:32.800745 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Lock "cinder-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2-delete_volume" acquired by "delete_volume" :: waited 0.020s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:06:32.846514 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:32.912548 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:06:32.913003 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Volume volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:06:32.920384 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] deleting rbd volume volume-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:06:32.922719 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Created reservations ['af10d721-8f11-4e34-9f94-6848cbb5397e', '2230b338-09aa-4219-9c33-88e6a2f62dcb', '55d2965d-941c-4d21-b30f-a4e8a2b2da2c', '9287b5b8-7f86-4084-893c-b0adce0420b4'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:06:32.985941 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:33.047712 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:33.143063 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:06:33.148719 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Deleted volume successfully.
Jul 02 21:06:33.167340 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-4bca53cf-b6f1-4284-897f-fe5d142917d5 req-567845eb-57ff-4410-866f-616a162d716e tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Lock "cinder-84fa484c-0fe4-4a8b-83a4-e46960aee0a8-delete_volume" released by "delete_volume" :: held 2.138s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:06:33.543622 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Created reservations ['ccb55d2f-65f0-481b-8acd-e5f6598619ba', '9c6b36fc-a4c9-46ea-84c1-378f516098b6', '904f5a9c-2154-4af6-849a-6ed29f6da6d6', 'ad888cb7-0290-4cf3-88a7-2c9e050487c5'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:06:33.625809 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:33.708524 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:33.778957 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:06:33.787273 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Deleted volume successfully.
Jul 02 21:06:33.804599 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-34a96271-a884-4095-a7ad-0e4c226b0788 tempest-LiveAutoBlockMigrationV225Test-1025963900 None] Lock "cinder-48d74eca-fcde-4742-9dbc-87d3e9f0e8c2-delete_volume" released by "delete_volume" :: held 1.004s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:06:34.147271 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:06:34.150534 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:34.249975 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:34.317283 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:06:43.797599 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=34,cinder:UPDATE=19,cinder:INSERT=4,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:06:49.227253 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=54,cinder:UPDATE=20,cinder:INSERT=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:06:54.775903 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:06:54.802368 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Flow 'volume_create_manager' (64319fb5-3c19-4ce4-9ea0-25c7120a3ef7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:06:54.811175 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b9a06858-fc5a-4ffe-ba1a-bea34abc2296) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:06:54.852790 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b9a06858-fc5a-4ffe-ba1a-bea34abc2296) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:06:54Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-1446879866',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=72680320-7036-4e2f-a778-86b690394823,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4b15f516c6a34343979d19a0938c232b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:06:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:06:55Z,use_quota=True,user_id='1fade188e2a14766ae95fe44143bca97',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:06:54.889944 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7fe3182a-dec6-462b-b496-08e1f1483955) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:06:54.898300 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (7fe3182a-dec6-462b-b496-08e1f1483955) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:06:54.906819 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f3abbff9-613d-4bcb-801e-0c749346b2e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:06:54.914346 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (f3abbff9-613d-4bcb-801e-0c749346b2e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '72680320-7036-4e2f-a778-86b690394823', 'volume_name': 'volume-72680320-7036-4e2f-a778-86b690394823', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:06:54.920605 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (27c420a8-afdb-405f-8e2e-e49d2bf161cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:06:54.925015 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (27c420a8-afdb-405f-8e2e-e49d2bf161cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:06:54.929905 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (949d8be7-f63c-48d2-9366-406b5eee177f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:06:54.931002 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Volume 72680320-7036-4e2f-a778-86b690394823: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-72680320-7036-4e2f-a778-86b690394823', 'volume_size': 1}
Jul 02 21:06:54.931507 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] creating volume 'volume-72680320-7036-4e2f-a778-86b690394823' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:06:54.932248 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:06:55.039380 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:06:55.039942 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:06:55.045026 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (949d8be7-f63c-48d2-9366-406b5eee177f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-72680320-7036-4e2f-a778-86b690394823', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:06:55.051182 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89b7c993-c456-4d0c-97aa-2be2545d128e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:06:55.064568 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Volume volume-72680320-7036-4e2f-a778-86b690394823 (72680320-7036-4e2f-a778-86b690394823): created successfully
Jul 02 21:06:55.069814 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (89b7c993-c456-4d0c-97aa-2be2545d128e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:06:55.077454 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Flow 'volume_create_manager' (64319fb5-3c19-4ce4-9ea0-25c7120a3ef7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:06:55.091620 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-3bee17a2-8c92-495e-be49-4cd70d4b3ad9 tempest-VolumesAdminNegativeTest-1317235312 None] Created volume successfully.
Jul 02 21:06:56.833308 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-48c6caca-6152-43eb-8ea8-ecf192f89005 req-8dfb9919-51fa-448d-8eb3-006915939005 tempest-VolumesAdminNegativeTest-1317235312 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:06:57.949515 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-48c6caca-6152-43eb-8ea8-ecf192f89005 req-8dfb9919-51fa-448d-8eb3-006915939005 tempest-VolumesAdminNegativeTest-1317235312 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.116s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:06:57.951189 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-48c6caca-6152-43eb-8ea8-ecf192f89005 req-8dfb9919-51fa-448d-8eb3-006915939005 tempest-VolumesAdminNegativeTest-1317235312 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-72680320-7036-4e2f-a778-86b690394823', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '72680320-7036-4e2f-a778-86b690394823', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:06:58.014008 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-48c6caca-6152-43eb-8ea8-ecf192f89005 req-8dfb9919-51fa-448d-8eb3-006915939005 tempest-VolumesAdminNegativeTest-1317235312 None] Connection info returned from driver {'name': 'volumes/volume-72680320-7036-4e2f-a778-86b690394823', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '72680320-7036-4e2f-a778-86b690394823', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'e8335cca-2d25-41ff-a7ff-df3c07de3e98'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:06:58.087548 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-48c6caca-6152-43eb-8ea8-ecf192f89005 req-8dfb9919-51fa-448d-8eb3-006915939005 tempest-VolumesAdminNegativeTest-1317235312 None] attachment_update completed successfully.
Jul 02 21:07:06.646355 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-a51fb2c8-756a-4739-be74-37e86ee26355 req-c233389b-df42-4309-a185-68891a3c1358 tempest-VolumesAdminNegativeTest-1317235312 None] Terminate volume connection completed successfully.
Jul 02 21:07:06.647029 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-a51fb2c8-756a-4739-be74-37e86ee26355 req-c233389b-df42-4309-a185-68891a3c1358 tempest-VolumesAdminNegativeTest-1317235312 None] Deleting attachment e8335cca-2d25-41ff-a7ff-df3c07de3e98. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:07:19.224899 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=24,cinder:UPDATE=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:07:29.230457 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=22,cinder:UPDATE=6,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:07:34.147959 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:07:34.149498 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:34.214053 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:34.282099 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:07:39.243713 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:07:40.180360 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:07:40.249117 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Flow 'volume_create_manager' (d9acb602-a89e-4109-9c4e-e5245d64451a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:07:40.271863 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (72b4d255-3755-453e-820e-77294dd89169) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:07:40.347636 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (72b4d255-3755-453e-820e-77294dd89169) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:07:40Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesAdminNegativeTest-volume-712561796',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=5e755bf8-405d-4050-ac15-a1ff84181eef,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='4b15f516c6a34343979d19a0938c232b',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:07:40Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:07:40Z,use_quota=True,user_id='1fade188e2a14766ae95fe44143bca97',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:07:40.364569 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (035890fa-835f-48cf-9ae0-86a6103d7672) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:07:40.383913 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (035890fa-835f-48cf-9ae0-86a6103d7672) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:07:40.392520 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e4c5d9fc-7851-4e14-9cfd-8e9469f9f427) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:07:40.406916 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e4c5d9fc-7851-4e14-9cfd-8e9469f9f427) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '5e755bf8-405d-4050-ac15-a1ff84181eef', 'volume_name': 'volume-5e755bf8-405d-4050-ac15-a1ff84181eef', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:07:40.412018 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1c7114ac-d8d5-4c27-ad01-10504951b34b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:07:40.460304 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1c7114ac-d8d5-4c27-ad01-10504951b34b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:07:40.471074 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7bfe76c5-14d7-46c2-adad-2a0ad2a5d900) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:07:40.473185 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Volume 5e755bf8-405d-4050-ac15-a1ff84181eef: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-5e755bf8-405d-4050-ac15-a1ff84181eef', 'volume_size': 1}
Jul 02 21:07:40.482701 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] creating volume 'volume-5e755bf8-405d-4050-ac15-a1ff84181eef' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:07:40.486230 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:40.695875 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:07:40.698365 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:07:40.706106 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (7bfe76c5-14d7-46c2-adad-2a0ad2a5d900) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-5e755bf8-405d-4050-ac15-a1ff84181eef', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:07:40.722094 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3ebf4cc-5b76-4f78-852d-3d23b3ce5754) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:07:40.744663 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Volume volume-5e755bf8-405d-4050-ac15-a1ff84181eef (5e755bf8-405d-4050-ac15-a1ff84181eef): created successfully
Jul 02 21:07:40.750820 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f3ebf4cc-5b76-4f78-852d-3d23b3ce5754) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:07:40.766044 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Flow 'volume_create_manager' (d9acb602-a89e-4109-9c4e-e5245d64451a) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:07:40.789936 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-3204c5ef-db5b-4132-a31e-da1c26970ee6 tempest-VolumesAdminNegativeTest-1317235312 None] Created volume successfully.
Jul 02 21:07:41.711250 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Acquiring lock "cinder-5e755bf8-405d-4050-ac15-a1ff84181eef-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:07:41.745061 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Lock "cinder-5e755bf8-405d-4050-ac15-a1ff84181eef-delete_volume" acquired by "delete_volume" :: waited 0.029s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:07:41.813870 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:41.945695 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:07:41.946525 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Volume volume-5e755bf8-405d-4050-ac15-a1ff84181eef is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:07:41.964669 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] deleting rbd volume volume-5e755bf8-405d-4050-ac15-a1ff84181eef {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:07:42.958426 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Created reservations ['3fe94dad-d379-4fc6-9a19-52d9979a8d67', 'a3f073bf-7b53-4a90-a291-985c02a4363c', '42ca0814-ae6f-43d6-af6d-c95f01b73646', 'f9e50713-4dcb-48f2-93c5-a42c63282329'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:07:43.089995 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:43.157216 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:43.212449 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:07:43.221217 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Deleted volume successfully.
Jul 02 21:07:43.245021 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-8ac3ad56-0984-49d7-889d-b9e538d25c82 tempest-VolumesAdminNegativeTest-1317235312 None] Lock "cinder-5e755bf8-405d-4050-ac15-a1ff84181eef-delete_volume" released by "delete_volume" :: held 1.504s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:07:50.203103 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Acquiring lock "cinder-72680320-7036-4e2f-a778-86b690394823-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:07:50.230537 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Lock "cinder-72680320-7036-4e2f-a778-86b690394823-delete_volume" acquired by "delete_volume" :: waited 0.028s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:07:50.308073 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:50.381261 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:07:50.381773 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Volume volume-72680320-7036-4e2f-a778-86b690394823 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:07:50.398127 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] deleting rbd volume volume-72680320-7036-4e2f-a778-86b690394823 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:07:50.736847 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Created reservations ['d80330af-1f8c-4fd4-bdab-0221337775b5', 'a4bf5876-d3e1-4150-8a52-da9d07d06495', '7a2165a8-8dfa-4bf5-b16e-763c96316105', '5a226f62-58b1-4e24-b6e1-677dc7962e0d'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:07:50.791425 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:50.838442 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:07:50.887348 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:07:50.891705 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Deleted volume successfully.
Jul 02 21:07:50.907231 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-62d0fd55-05f7-4ca1-b5aa-7a5dbc8955ec tempest-VolumesAdminNegativeTest-1317235312 None] Lock "cinder-72680320-7036-4e2f-a778-86b690394823-delete_volume" released by "delete_volume" :: held 0.677s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:08:00.896762 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=25,cinder:UPDATE=16,cinder:INSERT=5,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:08:09.253946 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=40,cinder:UPDATE=22,cinder:DELETE=2,cinder:INSERT=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:08:29.239484 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:08:31.002313 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=9,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:08:31.030350 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:08:31.064581 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Flow 'volume_create_manager' (31e81fa8-434e-4302-be5b-b8d505c610dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:08:31.072761 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ce86fc25-5887-4ab1-937e-b78c8f5fe48c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:08:31.092603 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (ce86fc25-5887-4ab1-937e-b78c8f5fe48c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:08:31Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachSCSIVolumeTestJSON-volume-1163619844',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=b72de476-9c34-4367-916c-e2e6f7f3a838,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='426d63ca5cfa48e19a7156d1b848447e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:08:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:08:31Z,use_quota=True,user_id='097020096c1c4ff2b67800be67cd708a',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:08:31.099089 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a3864191-4636-42fb-96bc-6473b1cf1089) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:08:31.105918 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a3864191-4636-42fb-96bc-6473b1cf1089) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:08:31.111712 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (939bc391-a7bb-4f03-8033-101bb9491e1f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:08:31.117353 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (939bc391-a7bb-4f03-8033-101bb9491e1f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'b72de476-9c34-4367-916c-e2e6f7f3a838', 'volume_name': 'volume-b72de476-9c34-4367-916c-e2e6f7f3a838', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:08:31.122769 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (69a9cd1b-0bb1-4792-aba6-acdd3b1f4ae1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:08:31.128428 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (69a9cd1b-0bb1-4792-aba6-acdd3b1f4ae1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:08:31.134265 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c08617e3-43fe-4399-b15e-72640d23b56a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:08:31.136088 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Volume b72de476-9c34-4367-916c-e2e6f7f3a838: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-b72de476-9c34-4367-916c-e2e6f7f3a838', 'volume_size': 1}
Jul 02 21:08:31.136554 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] creating volume 'volume-b72de476-9c34-4367-916c-e2e6f7f3a838' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:08:31.137522 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:08:31.271903 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:08:31.272399 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:08:31.276097 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c08617e3-43fe-4399-b15e-72640d23b56a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-b72de476-9c34-4367-916c-e2e6f7f3a838', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:08:31.280603 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b04b3b7-be23-450f-81f3-6fb5eab07992) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:08:31.302406 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Volume volume-b72de476-9c34-4367-916c-e2e6f7f3a838 (b72de476-9c34-4367-916c-e2e6f7f3a838): created successfully
Jul 02 21:08:31.306024 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (2b04b3b7-be23-450f-81f3-6fb5eab07992) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:08:31.321079 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Flow 'volume_create_manager' (31e81fa8-434e-4302-be5b-b8d505c610dd) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:08:31.333823 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-13153827-88c4-486d-9587-093a5a02ce4c tempest-AttachSCSIVolumeTestJSON-1715054636 None] Created volume successfully.
Jul 02 21:08:33.155334 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-6dcd0d70-8645-4b86-879e-fb5ec412fa23 req-6a877031-ce62-4ff4-b0b6-6f7052a331ca tempest-AttachSCSIVolumeTestJSON-1715054636 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:08:34.150536 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:08:34.151748 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:08:34.259550 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:08:34.333775 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:08:34.378625 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-6dcd0d70-8645-4b86-879e-fb5ec412fa23 req-6a877031-ce62-4ff4-b0b6-6f7052a331ca tempest-AttachSCSIVolumeTestJSON-1715054636 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.224s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:08:34.379391 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-6dcd0d70-8645-4b86-879e-fb5ec412fa23 req-6a877031-ce62-4ff4-b0b6-6f7052a331ca tempest-AttachSCSIVolumeTestJSON-1715054636 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-b72de476-9c34-4367-916c-e2e6f7f3a838', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'b72de476-9c34-4367-916c-e2e6f7f3a838', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:08:34.403939 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-6dcd0d70-8645-4b86-879e-fb5ec412fa23 req-6a877031-ce62-4ff4-b0b6-6f7052a331ca tempest-AttachSCSIVolumeTestJSON-1715054636 None] Connection info returned from driver {'name': 'volumes/volume-b72de476-9c34-4367-916c-e2e6f7f3a838', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'b72de476-9c34-4367-916c-e2e6f7f3a838', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'c2df4699-347a-4c9d-ad32-3cf559bbfd30'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:08:34.459916 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-6dcd0d70-8645-4b86-879e-fb5ec412fa23 req-6a877031-ce62-4ff4-b0b6-6f7052a331ca tempest-AttachSCSIVolumeTestJSON-1715054636 None] attachment_update completed successfully.
Jul 02 21:08:39.326089 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-6bb50798-a5bb-4152-a580-f3394a56b757 req-6ee3427c-a314-447e-9a82-378f2762c802 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Terminate volume connection completed successfully.
Jul 02 21:08:39.330330 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-6bb50798-a5bb-4152-a580-f3394a56b757 req-6ee3427c-a314-447e-9a82-378f2762c802 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Deleting attachment c2df4699-347a-4c9d-ad32-3cf559bbfd30. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:08:52.333768 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Acquiring lock "cinder-b72de476-9c34-4367-916c-e2e6f7f3a838-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:08:52.352114 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Lock "cinder-b72de476-9c34-4367-916c-e2e6f7f3a838-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:08:52.442436 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:08:52.510951 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:08:52.511394 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Volume volume-b72de476-9c34-4367-916c-e2e6f7f3a838 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:08:52.517285 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] deleting rbd volume volume-b72de476-9c34-4367-916c-e2e6f7f3a838 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:08:53.174544 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Created reservations ['0c2653c7-5ace-4b4f-85c4-1ebcd39dc5e4', 'b8914c89-97a9-460a-96d1-4d585f8ea825', '18bca3a9-59d5-43f7-af6c-bc57995613ea', 'e8addbdd-b106-496a-8ff6-2241930bc9ad'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:08:53.239102 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:08:53.322489 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:08:53.393094 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:08:53.405712 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Deleted volume successfully.
Jul 02 21:08:53.424608 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-e90c2776-453f-4efb-bba2-53b59bfacce7 tempest-AttachSCSIVolumeTestJSON-1715054636 None] Lock "cinder-b72de476-9c34-4367-916c-e2e6f7f3a838-delete_volume" released by "delete_volume" :: held 1.073s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:09:01.039696 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Acquiring lock "cinder-c2aa26fe-49ef-4733-8bdc-f5d9113ce416-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:09:01.055255 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Lock "cinder-c2aa26fe-49ef-4733-8bdc-f5d9113ce416-delete_volume" acquired by "delete_volume" :: waited 0.014s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:09:01.069909 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=37,cinder:UPDATE=17,cinder:INSERT=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:09:01.094896 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=32,cinder:DELETE=2,cinder:UPDATE=9,cinder:INSERT=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:09:01.108880 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:09:01.192175 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:09:01.192679 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Volume volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:09:01.201197 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] deleting rbd volume volume-c2aa26fe-49ef-4733-8bdc-f5d9113ce416 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:09:01.498658 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Created reservations ['cc6b8858-d383-4f0e-b2c9-03f286a4229b', '943bb6c6-b768-4846-8ac0-82494e5903fe', '268a885d-fd9b-43e1-bf6b-c6c5afdf5fd5', '4fa0ed2e-aa6f-4e81-9cd1-714ad38704c3'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:09:01.563357 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:09:01.616264 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:09:01.660387 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:09:01.666438 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Deleted volume successfully.
Jul 02 21:09:01.683347 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-55672661-1268-40de-8585-ae5178378088 tempest-DeleteServersTestJSON-1749805065 None] Lock "cinder-c2aa26fe-49ef-4733-8bdc-f5d9113ce416-delete_volume" released by "delete_volume" :: held 0.629s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:09:29.231920 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=10,cinder:UPDATE=12,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:09:34.152460 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:09:34.153129 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:09:34.235915 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:09:34.293326 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:09:39.246440 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=19,cinder:INSERT=4,cinder:UPDATE=6 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:10:04.414399 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:10:04.423162 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-2fea666d-7d99-4210-b634-a51b3cecc115 req-95d60cb5-f038-46b8-b2e2-204e49ac345d tempest-TaggedBootDevicesTest_v242-1793629843 None] Terminate volume connection completed successfully.
Jul 02 21:10:04.423587 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-2fea666d-7d99-4210-b634-a51b3cecc115 req-95d60cb5-f038-46b8-b2e2-204e49ac345d tempest-TaggedBootDevicesTest_v242-1793629843 None] Deleting attachment 6a1029c5-e65f-48cf-bb48-a696e1214fe1. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:10:04.611691 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-2fea666d-7d99-4210-b634-a51b3cecc115 req-54b61c5b-6c95-4aaa-881b-1a8f603f83ad tempest-TaggedBootDevicesTest_v242-1793629843 None] Terminate volume connection completed successfully.
Jul 02 21:10:04.612141 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-2fea666d-7d99-4210-b634-a51b3cecc115 req-54b61c5b-6c95-4aaa-881b-1a8f603f83ad tempest-TaggedBootDevicesTest_v242-1793629843 None] Deleting attachment a34492cc-0c39-4d38-be6b-702e55f3a576. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:10:04.850050 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-2fea666d-7d99-4210-b634-a51b3cecc115 req-bc229f2f-4d3a-4e29-97e4-461c9f4fe04c tempest-TaggedBootDevicesTest_v242-1793629843 None] Terminate volume connection completed successfully.
Jul 02 21:10:04.850468 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-2fea666d-7d99-4210-b634-a51b3cecc115 req-bc229f2f-4d3a-4e29-97e4-461c9f4fe04c tempest-TaggedBootDevicesTest_v242-1793629843 None] Deleting attachment 9fd01942-b0c2-401e-be58-0716ed6493ee. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:10:09.788759 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=14 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:10:09.805203 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:10:09.844249 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Flow 'volume_create_manager' (4b32b096-ab3e-4f64-9438-c07f9139fbc2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:10:09.853151 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9dd16072-2242-4d88-966f-d91133fb53c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:10:09.875972 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (9dd16072-2242-4d88-966f-d91133fb53c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:10:10Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-LiveMigrationTest-volume-728448949',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=655b7662-f602-41a5-8354-63f38f63b21d,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8c234eb1b17242368d279c164a6045d4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:10:10Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:10:10Z,use_quota=True,user_id='8b345a42784d4df79caab6d667117418',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:10:09.883419 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0e4f438c-1dba-4807-8c3f-441db4dd1b58) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:10:09.888139 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (0e4f438c-1dba-4807-8c3f-441db4dd1b58) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:10:09.923865 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4966d2d4-ef70-432a-8035-0afb8a3dabf5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:10:09.934051 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4966d2d4-ef70-432a-8035-0afb8a3dabf5) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '655b7662-f602-41a5-8354-63f38f63b21d', 'volume_name': 'volume-655b7662-f602-41a5-8354-63f38f63b21d', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:10:09.940171 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bc012bf4-c4a2-497d-b5fe-210a6e219f65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:10:09.945609 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bc012bf4-c4a2-497d-b5fe-210a6e219f65) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:10:09.951139 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca8a56e3-4dd3-482d-a618-fe8e00c0effa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:10:09.952833 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Volume 655b7662-f602-41a5-8354-63f38f63b21d: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-655b7662-f602-41a5-8354-63f38f63b21d', 'volume_size': 1}
Jul 02 21:10:09.953415 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] creating volume 'volume-655b7662-f602-41a5-8354-63f38f63b21d' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:10:09.954398 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:10.099200 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:10:10.099766 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:10:10.103724 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (ca8a56e3-4dd3-482d-a618-fe8e00c0effa) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-655b7662-f602-41a5-8354-63f38f63b21d', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:10:10.108114 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (072b0789-e4bd-4da0-b2dc-b393881dd756) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:10:10.123692 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Volume volume-655b7662-f602-41a5-8354-63f38f63b21d (655b7662-f602-41a5-8354-63f38f63b21d): created successfully
Jul 02 21:10:10.128741 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (072b0789-e4bd-4da0-b2dc-b393881dd756) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:10:10.135195 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Flow 'volume_create_manager' (4b32b096-ab3e-4f64-9438-c07f9139fbc2) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:10:10.153779 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-fb5f114d-2637-4450-a0ff-86459b79c075 tempest-LiveMigrationTest-1775332521 None] Created volume successfully.
Jul 02 21:10:11.715336 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-77e54e61-ee3e-40a2-8bd3-dad25ae499af req-23bb616d-7d4c-4b16-85ea-c6ae2cc4d0e0 tempest-LiveMigrationTest-1775332521 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:10:13.160060 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-77e54e61-ee3e-40a2-8bd3-dad25ae499af req-23bb616d-7d4c-4b16-85ea-c6ae2cc4d0e0 tempest-LiveMigrationTest-1775332521 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.444s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:10:13.160979 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-77e54e61-ee3e-40a2-8bd3-dad25ae499af req-23bb616d-7d4c-4b16-85ea-c6ae2cc4d0e0 tempest-LiveMigrationTest-1775332521 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-655b7662-f602-41a5-8354-63f38f63b21d', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '655b7662-f602-41a5-8354-63f38f63b21d', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:10:13.192130 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-77e54e61-ee3e-40a2-8bd3-dad25ae499af req-23bb616d-7d4c-4b16-85ea-c6ae2cc4d0e0 tempest-LiveMigrationTest-1775332521 None] Connection info returned from driver {'name': 'volumes/volume-655b7662-f602-41a5-8354-63f38f63b21d', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '655b7662-f602-41a5-8354-63f38f63b21d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'b3c69571-4ae5-4e1f-bf50-d55f67fc236f'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:10:13.263071 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-77e54e61-ee3e-40a2-8bd3-dad25ae499af req-23bb616d-7d4c-4b16-85ea-c6ae2cc4d0e0 tempest-LiveMigrationTest-1775332521 None] attachment_update completed successfully.
Jul 02 21:10:15.413436 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Acquiring lock "cinder-4af561d5-4d61-49c9-b2df-3812b0172ae2-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:10:15.443049 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-4af561d5-4d61-49c9-b2df-3812b0172ae2-delete_volume" acquired by "delete_volume" :: waited 0.029s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:10:15.535926 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:15.613925 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:10:15.614460 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-4af561d5-4d61-49c9-b2df-3812b0172ae2 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:10:15.629215 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] deleting rbd volume volume-4af561d5-4d61-49c9-b2df-3812b0172ae2 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:10:16.198699 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Created reservations ['54503600-1a96-44c4-8b0a-faeb1e5e42e2', 'b9534c3d-a37c-43b6-94b5-89806748b0d0', '10a77ed5-e559-47a2-92e1-84e9021c2270', '74c437b5-5252-47b6-95a2-19eea26d7c7b'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:10:16.321811 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:16.394780 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:16.489000 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:10:16.494082 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Deleted volume successfully.
Jul 02 21:10:16.518053 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-58d7c267-bba4-4a65-90c3-7ec3d998e9e9 tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-4af561d5-4d61-49c9-b2df-3812b0172ae2-delete_volume" released by "delete_volume" :: held 1.076s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:10:16.703632 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Acquiring lock "cinder-6da0916e-d49c-4e14-94a5-3cccd3b7c22c-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:10:16.730631 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-6da0916e-d49c-4e14-94a5-3cccd3b7c22c-delete_volume" acquired by "delete_volume" :: waited 0.027s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:10:16.794017 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:16.889505 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:10:16.890220 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:10:16.901039 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] deleting rbd volume volume-6da0916e-d49c-4e14-94a5-3cccd3b7c22c {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:10:17.447493 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Created reservations ['2f7952e9-378f-4c51-9bc0-e02fc7df95dc', '576ba1c1-677a-46bb-893b-aff9e40cb818', 'f47e05c1-9c60-421e-8332-ef6a7ea1ef01', '2f31146f-ffff-4636-b0c1-4d2df0a9859d'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:10:17.579094 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:17.649806 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:17.709557 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:10:17.714408 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Deleted volume successfully.
Jul 02 21:10:17.735981 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c682fde-547b-47d6-968a-e5c94bd62224 tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-6da0916e-d49c-4e14-94a5-3cccd3b7c22c-delete_volume" released by "delete_volume" :: held 1.005s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:10:17.931208 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Acquiring lock "cinder-0fd88419-3147-47fa-bed8-27c867972435-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:10:17.954735 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-0fd88419-3147-47fa-bed8-27c867972435-delete_volume" acquired by "delete_volume" :: waited 0.024s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:10:18.042905 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:18.175352 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:10:18.175668 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] volume has clone snapshot(s) {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1288}}
Jul 02 21:10:18.177356 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:10:18.383376 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Created reservations ['4d472629-8dc5-4f2f-995f-48a88c17a380', '55fafb5e-afa2-477a-9531-27f589072e82', '7e053a0c-9fdd-4002-9ca0-2ad8ff2d3641', '0f41f5b5-1e06-4483-9b80-9d8a0a5dab55'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:10:18.541905 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=58,cinder:UPDATE=36,cinder:INSERT=5,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:10:18.550339 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:18.639312 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:18.727951 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:10:18.734102 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Deleted volume successfully.
Jul 02 21:10:18.770364 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dcdda157-4b45-44bb-8b65-e848b081d872 tempest-TaggedBootDevicesTest_v242-1793629843 None] Lock "cinder-0fd88419-3147-47fa-bed8-27c867972435-delete_volume" released by "delete_volume" :: held 0.816s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:10:29.209395 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-23c9d46c-9852-4a34-b898-2a240eeecc0a tempest-LiveMigrationTest-855187274 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:10:30.224936 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-23c9d46c-9852-4a34-b898-2a240eeecc0a tempest-LiveMigrationTest-855187274 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.015s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:10:30.226548 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-23c9d46c-9852-4a34-b898-2a240eeecc0a tempest-LiveMigrationTest-855187274 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-655b7662-f602-41a5-8354-63f38f63b21d', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '655b7662-f602-41a5-8354-63f38f63b21d', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:10:30.250080 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-23c9d46c-9852-4a34-b898-2a240eeecc0a tempest-LiveMigrationTest-855187274 None] Connection info returned from driver {'name': 'volumes/volume-655b7662-f602-41a5-8354-63f38f63b21d', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '655b7662-f602-41a5-8354-63f38f63b21d', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '5cef1589-96f0-4d51-b76e-320ab3d7f427'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:10:30.329514 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-23c9d46c-9852-4a34-b898-2a240eeecc0a tempest-LiveMigrationTest-855187274 None] attachment_update completed successfully.
Jul 02 21:10:34.151289 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:10:34.153023 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:34.234207 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:10:34.310764 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:10:47.838384 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-bbd91704-0eb3-43d4-be33-36de1224bd5b tempest-LiveMigrationTest-855187274 None] Terminate volume connection completed successfully.
Jul 02 21:10:47.839033 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-6444786d-8638-481c-a9d7-2b3a2caabc51 req-bbd91704-0eb3-43d4-be33-36de1224bd5b tempest-LiveMigrationTest-855187274 None] Deleting attachment b3c69571-4ae5-4e1f-bf50-d55f67fc236f. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:10:49.234407 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=68,cinder:DELETE=3,cinder:UPDATE=22,cinder:INSERT=7 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:10:49.235225 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:UPDATE=3,cinder:SELECT=18 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:11:04.670233 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:11:04.704290 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Flow 'volume_create_manager' (c338ce15-0af9-47cb-82ff-5e194382b8e9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:11:04.715801 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7b5022a-e737-49be-b005-cd3925db0bec) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:04.749198 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c7b5022a-e737-49be-b005-cd3925db0bec) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:11:04Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotsTestJSON-volume-1095572702',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=f5accb98-0f42-4400-afaf-07719d59b072,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='01a157232bd54d46ad99f8655ebb40b2',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:11:05Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:11:05Z,use_quota=True,user_id='094610d1808a479b99ce0fe5c2bac033',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:04.771694 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d90e2768-7408-4e03-8637-79d128ae9280) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:04.775943 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (d90e2768-7408-4e03-8637-79d128ae9280) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:04.790585 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d95fefff-ac02-4f71-9800-34a395ea4d54) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:04.806898 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d95fefff-ac02-4f71-9800-34a395ea4d54) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'f5accb98-0f42-4400-afaf-07719d59b072', 'volume_name': 'volume-f5accb98-0f42-4400-afaf-07719d59b072', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:04.818969 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dd84230b-c4ee-4ff4-be3d-1df9512c17cd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:04.830379 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dd84230b-c4ee-4ff4-be3d-1df9512c17cd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:04.844871 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d0d051ba-2f2d-4e39-be83-b0d4c30d6c6e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:04.849844 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Volume f5accb98-0f42-4400-afaf-07719d59b072: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-f5accb98-0f42-4400-afaf-07719d59b072', 'volume_size': 1}
Jul 02 21:11:04.850274 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] creating volume 'volume-f5accb98-0f42-4400-afaf-07719d59b072' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:11:04.853016 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:05.038457 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-d7fb0084-c57c-45ef-8401-f536b9969ba5 req-47bffe8a-88ac-40d7-a130-f8a0adbc60fa tempest-LiveMigrationTest-1775332521 None] Terminate volume connection completed successfully.
Jul 02 21:11:05.038877 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-d7fb0084-c57c-45ef-8401-f536b9969ba5 req-47bffe8a-88ac-40d7-a130-f8a0adbc60fa tempest-LiveMigrationTest-1775332521 None] Deleting attachment 5cef1589-96f0-4d51-b76e-320ab3d7f427. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:11:05.078284 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:11:05.080157 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:11:05.086932 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d0d051ba-2f2d-4e39-be83-b0d4c30d6c6e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-f5accb98-0f42-4400-afaf-07719d59b072', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:05.095298 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b74fe7cb-1e52-495c-8f80-e0a3058296ad) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:05.121856 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Volume volume-f5accb98-0f42-4400-afaf-07719d59b072 (f5accb98-0f42-4400-afaf-07719d59b072): created successfully
Jul 02 21:11:05.133561 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (b74fe7cb-1e52-495c-8f80-e0a3058296ad) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:05.146305 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Flow 'volume_create_manager' (c338ce15-0af9-47cb-82ff-5e194382b8e9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:11:05.176395 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-1837dd84-a244-4808-8288-85bab3ff1b6c req-3f63ae9c-e955-4bf4-a5fc-6c33a833549f tempest-VolumesSnapshotsTestJSON-357739283 None] Created volume successfully.
Jul 02 21:11:06.845206 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-3f28603a-3fbf-4f12-9c3b-df6c68c758dd req-9b3ecaee-e04d-4e07-8c50-0fa6059454b0 tempest-VolumesSnapshotsTestJSON-357739283 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:07.892005 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-3f28603a-3fbf-4f12-9c3b-df6c68c758dd req-9b3ecaee-e04d-4e07-8c50-0fa6059454b0 tempest-VolumesSnapshotsTestJSON-357739283 None] Create snapshot completed successfully
Jul 02 21:11:08.043050 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:11:08.146606 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Flow 'volume_create_manager' (e1e07866-3b11-438f-809b-a40043de42d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:11:08.170294 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e0f015a3-15dd-43be-92ac-138f8a355914) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:08.247354 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e0f015a3-15dd-43be-92ac-138f8a355914) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:11:07Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-tempest.common.compute-volume-1607171327',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=a920da18-54ff-4013-859c-a7aab272a078,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='8c234eb1b17242368d279c164a6045d4',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:11:08Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:11:08Z,use_quota=True,user_id='8b345a42784d4df79caab6d667117418',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:08.270641 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ccadf7e8-2315-418e-b84a-b96226e94c0c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:08.290408 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ccadf7e8-2315-418e-b84a-b96226e94c0c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:08.301611 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb8c07a7-019a-4d69-9fa3-e3954700013d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:08.519176 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (bb8c07a7-019a-4d69-9fa3-e3954700013d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': 'a920da18-54ff-4013-859c-a7aab272a078', 'volume_name': 'volume-a920da18-54ff-4013-859c-a7aab272a078', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f80729ebaf0>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:08.548888 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ded53a8f-4e3b-4b61-82ff-f77cf7b858b6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:08.590590 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ded53a8f-4e3b-4b61-82ff-f77cf7b858b6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:08.602404 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aad51e2e-5152-410d-ad88-780d77e2535f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:08.605238 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Volume a920da18-54ff-4013-859c-a7aab272a078: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-a920da18-54ff-4013-859c-a7aab272a078', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f80729ebaf0>}
Jul 02 21:11:08.609827 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Cloning a920da18-54ff-4013-859c-a7aab272a078 from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=124013) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:11:08.612623 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:11:08.638140 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.026s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:11:08.675027 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] Acquiring lock "cinder-c745f9d7-d8c8-4d01-b052-cc75922b374e-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:11:08.694681 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 11, 8, 645954)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:11:08.700811 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:11:08.701391 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Found cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}}
Jul 02 21:11:08.715285 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 0.077s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:11:08.719696 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:11:08.739236 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 11, 8, 729002)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:11:08.739732 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:11:08.740292 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Creating from source image-volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45 {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}}
Jul 02 21:11:08.784954 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] Lock "cinder-c745f9d7-d8c8-4d01-b052-cc75922b374e-delete_snapshot" acquired by "delete_snapshot" :: waited 0.110s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:11:08.789327 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:08.912378 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:09.045340 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] creating snapshot='volume-a920da18-54ff-4013-859c-a7aab272a078.clone_snap' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:11:09.918566 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] cloning 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45@volume-a920da18-54ff-4013-859c-a7aab272a078.clone_snap' to 'volume-a920da18-54ff-4013-859c-a7aab272a078' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}}
Jul 02 21:11:10.146103 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435.deleted is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:11:10.147847 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:11:10.149987 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:11:10.203350 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] clone created successfully {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}}
Jul 02 21:11:10.204449 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.volume_utils [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Marking volume a920da18-54ff-4013-859c-a7aab272a078 as bootable. {{(pid=124013) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}}
Jul 02 21:11:10.221918 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Copying metadata from image e482b755-1c1a-4758-a7c0-b3edf1610541 to a920da18-54ff-4013-859c-a7aab272a078. {{(pid=124013) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}}
Jul 02 21:11:10.222720 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Creating volume glance metadata for volume a920da18-54ff-4013-859c-a7aab272a078 backed by image e482b755-1c1a-4758-a7c0-b3edf1610541 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=124013) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}}
Jul 02 21:11:10.383419 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (aad51e2e-5152-410d-ad88-780d77e2535f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a920da18-54ff-4013-859c-a7aab272a078', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f80729ebaf0>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:10.418034 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (55693569-ac67-47c8-a2f1-dffc7f6005d4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:11:10.508810 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Volume volume-a920da18-54ff-4013-859c-a7aab272a078 (a920da18-54ff-4013-859c-a7aab272a078): created successfully
Jul 02 21:11:10.526848 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (55693569-ac67-47c8-a2f1-dffc7f6005d4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:11:10.534339 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Flow 'volume_create_manager' (e1e07866-3b11-438f-809b-a40043de42d4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:11:10.554323 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-569f1445-f600-4c23-824d-ab0e8d4abf20 tempest-LiveMigrationTest-1775332521 None] Created volume successfully.
Jul 02 21:11:11.041493 ncdv-master-1 cinder-volume[124013]: WARNING cinder.quota [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:11:11.078447 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] Created reservations ['87e743cc-0cf2-435a-979e-a1534b3501ce', '9994ff22-41e4-431e-9675-0e8a83773c16', '9c915e13-9649-49c5-8dd8-f16610e6f845', 'bd2fa3a3-982c-4eb0-8d96-13e53ca35f09'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:11:11.186828 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] Delete snapshot completed successfully.
Jul 02 21:11:11.191396 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-4c61991d-515e-4ea4-b4a4-7213c4dccb2b req-321cdefd-5713-41be-9f22-c5c4b2cdb3ea tempest-VolumesSnapshotsTestJSON-357739283 None] Lock "cinder-c745f9d7-d8c8-4d01-b052-cc75922b374e-delete_snapshot" released by "delete_snapshot" :: held 2.406s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:11:14.881770 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Acquiring lock "cinder-f5accb98-0f42-4400-afaf-07719d59b072-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:11:14.903092 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Lock "cinder-f5accb98-0f42-4400-afaf-07719d59b072-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:11:15.023832 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:15.123630 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:11:15.124354 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Volume volume-f5accb98-0f42-4400-afaf-07719d59b072 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:11:15.140508 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] deleting rbd volume volume-f5accb98-0f42-4400-afaf-07719d59b072 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:11:15.653648 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Created reservations ['58681d6e-468c-4d1a-bb1f-8ab723d36d60', '131c7af7-28fc-4aa9-a0b5-6b6985229a75', '08e96e34-1513-4df5-b17f-e8a5bac6bb68', '93da8f26-9b50-4580-b331-c7597770f246'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:11:15.835935 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:15.989074 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:16.106790 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:11:16.119751 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Deleted volume successfully.
Jul 02 21:11:16.143334 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-5b71ed23-52c6-4963-9b15-3fc9b42d2a2d req-b798a860-964f-4d60-a228-8748aa2dabf9 tempest-VolumesSnapshotsTestJSON-357739283 None] Lock "cinder-f5accb98-0f42-4400-afaf-07719d59b072-delete_volume" released by "delete_volume" :: held 1.240s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:11:17.843921 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-d50dfc1c-0736-403e-8196-8171554238f0 req-05d9043b-7c1a-4ade-a8f5-d27967c3ed51 tempest-LiveMigrationTest-1775332521 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:11:19.230166 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-d50dfc1c-0736-403e-8196-8171554238f0 req-05d9043b-7c1a-4ade-a8f5-d27967c3ed51 tempest-LiveMigrationTest-1775332521 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.387s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:11:19.231011 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-d50dfc1c-0736-403e-8196-8171554238f0 req-05d9043b-7c1a-4ade-a8f5-d27967c3ed51 tempest-LiveMigrationTest-1775332521 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a920da18-54ff-4013-859c-a7aab272a078', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'a920da18-54ff-4013-859c-a7aab272a078', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:11:19.274578 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-d50dfc1c-0736-403e-8196-8171554238f0 req-05d9043b-7c1a-4ade-a8f5-d27967c3ed51 tempest-LiveMigrationTest-1775332521 None] Connection info returned from driver {'name': 'volumes/volume-a920da18-54ff-4013-859c-a7aab272a078', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a920da18-54ff-4013-859c-a7aab272a078', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '459d0bb8-c0b8-42ef-a02f-c8147780090e'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:11:19.303915 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=53,cinder:UPDATE=21,cinder:INSERT=12,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:11:19.342073 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=66,cinder:UPDATE=23,cinder:DELETE=3,cinder:INSERT=8 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:11:19.362410 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-d50dfc1c-0736-403e-8196-8171554238f0 req-05d9043b-7c1a-4ade-a8f5-d27967c3ed51 tempest-LiveMigrationTest-1775332521 None] attachment_update completed successfully.
Jul 02 21:11:29.930040 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:11:34.155764 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:11:34.156887 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:34.250362 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:11:34.309795 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:11:46.913975 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-b8e287b0-c128-4a20-9c46-addfcf3341a7 tempest-LiveMigrationTest-855187274 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:11:49.087005 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-b8e287b0-c128-4a20-9c46-addfcf3341a7 tempest-LiveMigrationTest-855187274 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 2.173s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:11:49.087853 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-b8e287b0-c128-4a20-9c46-addfcf3341a7 tempest-LiveMigrationTest-855187274 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a920da18-54ff-4013-859c-a7aab272a078', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'a920da18-54ff-4013-859c-a7aab272a078', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:11:49.143510 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-b8e287b0-c128-4a20-9c46-addfcf3341a7 tempest-LiveMigrationTest-855187274 None] Connection info returned from driver {'name': 'volumes/volume-a920da18-54ff-4013-859c-a7aab272a078', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a920da18-54ff-4013-859c-a7aab272a078', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'aec3d871-3ec5-4b36-8c64-4c9824d815ba'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:11:49.268657 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-b8e287b0-c128-4a20-9c46-addfcf3341a7 tempest-LiveMigrationTest-855187274 None] attachment_update completed successfully.
Jul 02 21:11:49.934968 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=22,cinder:UPDATE=5 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:11:59.994889 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=12,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:12:01.456116 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-5d905317-42d6-46e8-8df7-139662bd1e8e tempest-LiveMigrationTest-855187274 None] Terminate volume connection completed successfully.
Jul 02 21:12:01.457315 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-1670f79b-83da-488f-98dd-2dc9f4b85b5d req-5d905317-42d6-46e8-8df7-139662bd1e8e tempest-LiveMigrationTest-855187274 None] Deleting attachment 459d0bb8-c0b8-42ef-a02f-c8147780090e. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:12:09.774371 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-04b03a67-1c0d-433a-bab1-c2e4686d763f tempest-LiveMigrationTest-1775332521 None] Terminate volume connection completed successfully.
Jul 02 21:12:09.775037 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-04b03a67-1c0d-433a-bab1-c2e4686d763f tempest-LiveMigrationTest-1775332521 None] Deleting attachment aec3d871-3ec5-4b36-8c64-4c9824d815ba. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:12:10.047272 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] Acquiring lock "cinder-a920da18-54ff-4013-859c-a7aab272a078-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:12:10.059493 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] Lock "cinder-a920da18-54ff-4013-859c-a7aab272a078-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:12:10.113486 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:10.405550 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:12:10.433773 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] deleting rbd volume volume-a920da18-54ff-4013-859c-a7aab272a078 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:12:10.949326 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] volume is a clone so cleaning references {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1346}}
Jul 02 21:12:11.025039 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] deleting parent snapshot volume-a920da18-54ff-4013-859c-a7aab272a078.clone_snap {{(pid=124013) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}}
Jul 02 21:12:11.656704 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Acquiring lock "cinder-655b7662-f602-41a5-8354-63f38f63b21d-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:12:11.692002 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Lock "cinder-655b7662-f602-41a5-8354-63f38f63b21d-delete_volume" acquired by "delete_volume" :: waited 0.035s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:12:11.888817 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] Created reservations ['d9bd4829-abeb-42a1-b5cc-7defab615562', '6d846a98-7df0-48a1-84e8-9db4dfbe85db', '14f2b971-49de-487d-967e-20ea2c839da6', 'a275a589-7c1d-4815-98ac-ff9f9c942042'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:12:11.928526 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:12.073388 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:12:12.074079 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Volume volume-655b7662-f602-41a5-8354-63f38f63b21d is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:12:12.099573 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] deleting rbd volume volume-655b7662-f602-41a5-8354-63f38f63b21d {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:12:12.182882 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:12.271348 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:12.412159 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:12:12.426296 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] Deleted volume successfully.
Jul 02 21:12:12.475140 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [req-276933dd-4295-4dd1-84f8-4f8bce8be5ee req-3a7533b5-168b-471b-af3c-157463f88677 tempest-LiveMigrationTest-1775332521 None] Lock "cinder-a920da18-54ff-4013-859c-a7aab272a078-delete_volume" released by "delete_volume" :: held 2.416s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:12:12.706927 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Created reservations ['1bb0c277-e951-46bf-80ee-b305c8359a11', '39ed92dc-ce91-499a-ac0b-e834523c9365', '629fb2d7-60d9-456c-967f-9b0a833515a4', 'd85845ba-a9cc-4dbd-a682-d458099c8c3a'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:12:12.836332 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:12.922839 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:13.037117 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:12:13.047787 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Deleted volume successfully.
Jul 02 21:12:13.104264 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-1c2cd845-7fba-41fb-af79-485ad45c9203 tempest-LiveMigrationTest-1775332521 None] Lock "cinder-655b7662-f602-41a5-8354-63f38f63b21d-delete_volume" released by "delete_volume" :: held 1.412s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:12:23.066101 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=26,cinder:UPDATE=17,cinder:INSERT=5 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:12:29.934531 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=40,cinder:UPDATE=18,cinder:INSERT=3,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:12:34.160647 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:12:34.161495 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:34.246505 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:12:34.337428 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:12:39.942614 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=2,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:12:59.942258 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:13:09.949371 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:13:29.945121 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:13:30.765454 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=8,cinder:UPDATE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:13:30.775750 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:13:30.795839 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Flow 'volume_create_manager' (01e2ac11-0e2d-4997-861c-c1f70e2d0710) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:13:30.802592 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1d5749af-501a-406e-a96c-380747b15226) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:13:30.819329 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1d5749af-501a-406e-a96c-380747b15226) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:13:30Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-243830127',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=aa1b5e5c-92f9-4ce4-b42d-2aeafada2342,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f644f2d1c56c43cf851deb020af7161e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:13:31Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:13:31Z,use_quota=True,user_id='984631f5a77a48c5a9619e33e608dd59',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:13:30.824828 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c56f622c-21e5-472e-bc01-8d27e3baff49) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:13:30.828977 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (c56f622c-21e5-472e-bc01-8d27e3baff49) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:13:30.834022 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (465d9109-64e0-4d1b-9741-5f76986c666e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:13:30.838374 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (465d9109-64e0-4d1b-9741-5f76986c666e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'volume_name': 'volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:13:30.843167 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fb0a3805-a399-46e9-88c5-e847ede06a67) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:13:30.848478 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (fb0a3805-a399-46e9-88c5-e847ede06a67) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:13:30.854821 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6a67b57f-7eff-483f-a16f-2b3786215541) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:13:30.856297 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume aa1b5e5c-92f9-4ce4-b42d-2aeafada2342: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'volume_size': 1}
Jul 02 21:13:30.856868 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] creating volume 'volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:13:30.857665 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:13:30.933207 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:13:30.933751 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:13:30.937485 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (6a67b57f-7eff-483f-a16f-2b3786215541) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:13:30.941966 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a20502e9-4c28-45e1-baf1-0a9c5e251539) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:13:30.953754 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342 (aa1b5e5c-92f9-4ce4-b42d-2aeafada2342): created successfully
Jul 02 21:13:30.962419 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a20502e9-4c28-45e1-baf1-0a9c5e251539) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:13:30.969655 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Flow 'volume_create_manager' (01e2ac11-0e2d-4997-861c-c1f70e2d0710) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:13:30.982914 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-a9d974fa-649a-4c9f-a291-5cc187f60699 tempest-AttachVolumeShelveTestJSON-1224937377 None] Created volume successfully.
Jul 02 21:13:34.160851 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:13:34.161703 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:13:34.200939 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:13:34.267341 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:13:40.990177 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=3,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:13:49.939789 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=3,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:14:09.942194 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=5,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:14:22.464605 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=7,cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:14:22.487227 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:14:22.588920 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Flow 'volume_create_manager' (28cf083b-1374-43d6-8607-b340f67375b4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:14:22.617698 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c5b7e78-f08e-4428-9a99-93e55dc45037) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:22.663994 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0c5b7e78-f08e-4428-9a99-93e55dc45037) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:14:22Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-1342191754',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=4f464256-167f-499c-a620-977a48ba0220,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b7da173babe7403fa85f5aab929e3068',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:14:22Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:14:22Z,use_quota=True,user_id='0f5b2e7e4efe478abdc85fc51fa064de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:22.693176 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6e3f9380-6e4d-4d8f-ae26-332c125528e8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:22.697640 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6e3f9380-6e4d-4d8f-ae26-332c125528e8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:22.712088 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4beaaca7-7843-48e7-97b3-6e668908f57a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:22.912694 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4beaaca7-7843-48e7-97b3-6e668908f57a) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '4f464256-167f-499c-a620-977a48ba0220', 'volume_name': 'volume-4f464256-167f-499c-a620-977a48ba0220', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072aa1f10>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:22.948963 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4f259b2c-90ca-4300-a7d4-b54fd2864152) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:22.966100 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4f259b2c-90ca-4300-a7d4-b54fd2864152) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:22.983267 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (205c8709-5248-4415-ba8d-91bfefae92a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:22.984451 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Volume 4f464256-167f-499c-a620-977a48ba0220: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-4f464256-167f-499c-a620-977a48ba0220', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072aa1f10>}
Jul 02 21:14:22.985029 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Cloning 4f464256-167f-499c-a620-977a48ba0220 from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=124013) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:14:22.985776 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:14:23.060833 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.075s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:14:23.139960 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 14, 23, 73908)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:14:23.140673 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:14:23.141244 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Found cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}}
Jul 02 21:14:23.147383 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 0.087s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:14:23.148726 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:14:23.156978 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 14, 23, 153413)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:14:23.157717 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:14:23.158250 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Creating from source image-volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45 {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}}
Jul 02 21:14:23.181571 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:14:23.344233 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] creating snapshot='volume-4f464256-167f-499c-a620-977a48ba0220.clone_snap' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:14:24.330670 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] cloning 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45@volume-4f464256-167f-499c-a620-977a48ba0220.clone_snap' to 'volume-4f464256-167f-499c-a620-977a48ba0220' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}}
Jul 02 21:14:24.615051 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Volume volume-0fd88419-3147-47fa-bed8-27c867972435.deleted is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:14:24.620443 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:14:24.621359 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:14:24.688736 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] clone created successfully {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:858}}
Jul 02 21:14:24.689772 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.volume_utils [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Marking volume 4f464256-167f-499c-a620-977a48ba0220 as bootable. {{(pid=124013) enable_bootable_flag /opt/stack/cinder/cinder/volume/volume_utils.py:1131}}
Jul 02 21:14:24.708862 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Copying metadata from image e482b755-1c1a-4758-a7c0-b3edf1610541 to 4f464256-167f-499c-a620-977a48ba0220. {{(pid=124013) _handle_bootable_volume_glance_meta /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:451}}
Jul 02 21:14:24.709538 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Creating volume glance metadata for volume 4f464256-167f-499c-a620-977a48ba0220 backed by image e482b755-1c1a-4758-a7c0-b3edf1610541 with: {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': '', 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_name': 'ubuntu-16', 'checksum': '318f990048c8a194829e52141534610d', 'container_format': 'bare', 'disk_format': 'vhdx', 'min_disk': 0, 'min_ram': 0, 'size': 645922816}. {{(pid=124013) _capture_volume_image_metadata /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:681}}
Jul 02 21:14:24.855712 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (205c8709-5248-4415-ba8d-91bfefae92a9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4f464256-167f-499c-a620-977a48ba0220', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072aa1f10>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:24.863403 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cc618d18-ea88-4e99-b6fe-dd37d7c6c5fa) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:24.904599 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Volume volume-4f464256-167f-499c-a620-977a48ba0220 (4f464256-167f-499c-a620-977a48ba0220): created successfully
Jul 02 21:14:24.924085 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (cc618d18-ea88-4e99-b6fe-dd37d7c6c5fa) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:24.958784 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Flow 'volume_create_manager' (28cf083b-1374-43d6-8607-b340f67375b4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:14:24.978447 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-bbec1967-9360-41a8-b812-6ba8efbbd180 tempest-TaggedBootDevicesTest-1312932753 None] Created volume successfully.
Jul 02 21:14:26.786826 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:14:26.844437 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Flow 'volume_create_manager' (c82140ae-3486-454c-89e0-fbf05473e5c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:14:26.866820 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (230f1746-dc26-47a4-9899-62cd8f8fe442) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:26.904803 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (230f1746-dc26-47a4-9899-62cd8f8fe442) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:14:26Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-545390588',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=a22e0b4d-265f-434c-bfd9-e859326dc395,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b7da173babe7403fa85f5aab929e3068',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:14:27Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:14:27Z,use_quota=True,user_id='0f5b2e7e4efe478abdc85fc51fa064de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:26.936106 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6647a40d-bde8-4412-9bf7-6c0d1116546f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:26.960893 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (6647a40d-bde8-4412-9bf7-6c0d1116546f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:26.976508 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c61aa18d-45e9-487e-8518-7abf9d3c543b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:27.003850 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c61aa18d-45e9-487e-8518-7abf9d3c543b) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a22e0b4d-265f-434c-bfd9-e859326dc395', 'volume_name': 'volume-a22e0b4d-265f-434c-bfd9-e859326dc395', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:27.014481 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (568133d5-e268-40f3-a724-8bfd81ea4c4c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:27.018788 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (568133d5-e268-40f3-a724-8bfd81ea4c4c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:27.028615 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (317113aa-2e0d-4469-97d9-7b20fc859a16) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:27.036085 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Volume a22e0b4d-265f-434c-bfd9-e859326dc395: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a22e0b4d-265f-434c-bfd9-e859326dc395', 'volume_size': 1}
Jul 02 21:14:27.036853 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] creating volume 'volume-a22e0b4d-265f-434c-bfd9-e859326dc395' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:14:27.037782 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:14:27.209856 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:14:27.209856 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:14:27.214359 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (317113aa-2e0d-4469-97d9-7b20fc859a16) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a22e0b4d-265f-434c-bfd9-e859326dc395', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:27.225107 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ee57dd76-dc99-4297-8295-d6d3311f9805) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:27.253589 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Volume volume-a22e0b4d-265f-434c-bfd9-e859326dc395 (a22e0b4d-265f-434c-bfd9-e859326dc395): created successfully
Jul 02 21:14:27.263089 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (ee57dd76-dc99-4297-8295-d6d3311f9805) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:27.272001 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Flow 'volume_create_manager' (c82140ae-3486-454c-89e0-fbf05473e5c1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:14:27.293904 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-43e844a4-eec0-4fc0-a4cb-0ac4eebd4c48 tempest-TaggedBootDevicesTest-1312932753 None] Created volume successfully.
Jul 02 21:14:28.491548 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:14:28.527325 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Flow 'volume_create_manager' (798b7c14-f78d-4faf-921d-9363597d7de4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:14:28.539379 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c6b10cfa-f65c-4944-ac20-71938637de00) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:28.584364 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (c6b10cfa-f65c-4944-ac20-71938637de00) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:14:28Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-TaggedBootDevicesTest-volume-2003741301',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=6593f24e-5231-490f-8bfd-665de85ebc5b,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='b7da173babe7403fa85f5aab929e3068',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:14:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:14:28Z,use_quota=True,user_id='0f5b2e7e4efe478abdc85fc51fa064de',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:28.607931 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9d59d422-9afa-4659-acbb-4580520898d1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:28.623586 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (9d59d422-9afa-4659-acbb-4580520898d1) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:28.629665 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b51590c2-f53d-45e9-994e-7a4687885734) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:28.634629 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (b51590c2-f53d-45e9-994e-7a4687885734) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6593f24e-5231-490f-8bfd-665de85ebc5b', 'volume_name': 'volume-6593f24e-5231-490f-8bfd-665de85ebc5b', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:28.649493 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bdd1875f-5110-4d0f-aa86-6b293e4ff928) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:28.663686 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (bdd1875f-5110-4d0f-aa86-6b293e4ff928) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:28.668555 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (20b72ded-9db4-42e4-b6db-6e63fc57f3b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:28.669836 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Volume 6593f24e-5231-490f-8bfd-665de85ebc5b: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6593f24e-5231-490f-8bfd-665de85ebc5b', 'volume_size': 1}
Jul 02 21:14:28.670393 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] creating volume 'volume-6593f24e-5231-490f-8bfd-665de85ebc5b' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:14:28.671186 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:14:28.853643 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:14:28.856882 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:14:28.861471 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (20b72ded-9db4-42e4-b6db-6e63fc57f3b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6593f24e-5231-490f-8bfd-665de85ebc5b', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:28.867380 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9294468a-1d1b-486b-a78f-23eec54d7627) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:14:28.886807 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Volume volume-6593f24e-5231-490f-8bfd-665de85ebc5b (6593f24e-5231-490f-8bfd-665de85ebc5b): created successfully
Jul 02 21:14:28.900726 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (9294468a-1d1b-486b-a78f-23eec54d7627) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:14:28.917188 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Flow 'volume_create_manager' (798b7c14-f78d-4faf-921d-9363597d7de4) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:14:28.937701 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-935c1772-8747-4e13-97d1-61557849fb97 tempest-TaggedBootDevicesTest-1312932753 None] Created volume successfully.
Jul 02 21:14:34.164691 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:14:34.165383 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:14:34.229316 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:14:34.327167 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:14:37.581216 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-ca9bd5b8-643f-4430-9128-dd97ed90f5fc req-dc85317c-4f32-46e0-bb22-14b19bf62a69 tempest-AttachVolumeShelveTestJSON-1224937377 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:14:39.135985 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-ca9bd5b8-643f-4430-9128-dd97ed90f5fc req-dc85317c-4f32-46e0-bb22-14b19bf62a69 tempest-AttachVolumeShelveTestJSON-1224937377 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.554s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:14:39.138857 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-ca9bd5b8-643f-4430-9128-dd97ed90f5fc req-dc85317c-4f32-46e0-bb22-14b19bf62a69 tempest-AttachVolumeShelveTestJSON-1224937377 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:14:39.184176 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-ca9bd5b8-643f-4430-9128-dd97ed90f5fc req-dc85317c-4f32-46e0-bb22-14b19bf62a69 tempest-AttachVolumeShelveTestJSON-1224937377 None] Connection info returned from driver {'name': 'volumes/volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'aa1b5e5c-92f9-4ce4-b42d-2aeafada2342', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'ebfbfdcc-832d-4238-bc57-3657bca2455f'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:14:39.364289 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-ca9bd5b8-643f-4430-9128-dd97ed90f5fc req-dc85317c-4f32-46e0-bb22-14b19bf62a69 tempest-AttachVolumeShelveTestJSON-1224937377 None] attachment_update completed successfully.
Jul 02 21:14:39.977907 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=30,cinder:UPDATE=10,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:14:49.980461 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:UPDATE=8,cinder:SELECT=43,cinder:INSERT=12,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:14:59.705739 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-f3bf8474-25b1-4623-8cab-d751f4147b97 tempest-TaggedBootDevicesTest-1312932753 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:15:01.246538 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-f3bf8474-25b1-4623-8cab-d751f4147b97 tempest-TaggedBootDevicesTest-1312932753 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.541s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:15:01.248249 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-f3bf8474-25b1-4623-8cab-d751f4147b97 tempest-TaggedBootDevicesTest-1312932753 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4f464256-167f-499c-a620-977a48ba0220', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '4f464256-167f-499c-a620-977a48ba0220', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:15:01.290879 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-f3bf8474-25b1-4623-8cab-d751f4147b97 tempest-TaggedBootDevicesTest-1312932753 None] Connection info returned from driver {'name': 'volumes/volume-4f464256-167f-499c-a620-977a48ba0220', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '4f464256-167f-499c-a620-977a48ba0220', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': 'aec6e3a4-21cd-42c2-9071-9bad04197d87'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:15:01.372439 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-f3bf8474-25b1-4623-8cab-d751f4147b97 tempest-TaggedBootDevicesTest-1312932753 None] attachment_update completed successfully.
Jul 02 21:15:01.940204 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-b7173098-041b-4cb3-a3ba-551ceeb0dbaf tempest-TaggedBootDevicesTest-1312932753 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:15:03.772222 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-b7173098-041b-4cb3-a3ba-551ceeb0dbaf tempest-TaggedBootDevicesTest-1312932753 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.837s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:15:03.773842 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-b7173098-041b-4cb3-a3ba-551ceeb0dbaf tempest-TaggedBootDevicesTest-1312932753 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-a22e0b4d-265f-434c-bfd9-e859326dc395', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'a22e0b4d-265f-434c-bfd9-e859326dc395', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:15:03.848313 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-b7173098-041b-4cb3-a3ba-551ceeb0dbaf tempest-TaggedBootDevicesTest-1312932753 None] Connection info returned from driver {'name': 'volumes/volume-a22e0b4d-265f-434c-bfd9-e859326dc395', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': 'a22e0b4d-265f-434c-bfd9-e859326dc395', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '012237dd-ce78-464f-b331-baf72883d8c7'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:15:03.957789 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-b7173098-041b-4cb3-a3ba-551ceeb0dbaf tempest-TaggedBootDevicesTest-1312932753 None] attachment_update completed successfully.
Jul 02 21:15:04.701982 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-6528e6d6-07cc-4e9b-bbab-e6e6d75db298 tempest-TaggedBootDevicesTest-1312932753 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:15:06.041355 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-6528e6d6-07cc-4e9b-bbab-e6e6d75db298 tempest-TaggedBootDevicesTest-1312932753 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.339s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:15:06.042707 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-6528e6d6-07cc-4e9b-bbab-e6e6d75db298 tempest-TaggedBootDevicesTest-1312932753 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-6593f24e-5231-490f-8bfd-665de85ebc5b', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '6593f24e-5231-490f-8bfd-665de85ebc5b', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:15:06.077567 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-6528e6d6-07cc-4e9b-bbab-e6e6d75db298 tempest-TaggedBootDevicesTest-1312932753 None] Connection info returned from driver {'name': 'volumes/volume-6593f24e-5231-490f-8bfd-665de85ebc5b', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '6593f24e-5231-490f-8bfd-665de85ebc5b', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '39bcae2c-ba03-4f6a-8f5b-387b1183c4b1'} {{(pid=124013) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:15:06.194762 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-870e88ac-0514-4753-932d-cfa8b5e50d0c req-6528e6d6-07cc-4e9b-bbab-e6e6d75db298 tempest-TaggedBootDevicesTest-1312932753 None] attachment_update completed successfully.
Jul 02 21:15:19.944992 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=43,cinder:UPDATE=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:15:29.945271 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=33,cinder:UPDATE=9 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:15:34.163728 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:15:34.165107 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:15:34.243000 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:15:34.309656 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:15:53.499798 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=8,cinder:UPDATE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:15:53.515205 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-563cdf8b-de21-4a34-b04d-6b4385e8d7cb tempest-GroupSnapshotsV319Test-131159047 None] Group tempest-GroupSnapshotsV319Test-Group-1758239301: creating
Jul 02 21:15:53.564915 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-563cdf8b-de21-4a34-b04d-6b4385e8d7cb tempest-GroupSnapshotsV319Test-131159047 None] Group tempest-GroupSnapshotsV319Test-Group-1758239301: created successfully
Jul 02 21:15:53.577134 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-563cdf8b-de21-4a34-b04d-6b4385e8d7cb tempest-GroupSnapshotsV319Test-131159047 None] Create group completed successfully.
Jul 02 21:15:55.224097 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:15:55.261056 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Flow 'volume_create_manager' (1d5db8f0-9186-4830-8c0e-28dcce364d0e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:15:55.268789 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e4c8d39-460f-484a-8633-b94c4814e8ed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:15:55.286481 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (3e4c8d39-460f-484a-8633-b94c4814e8ed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:15:55Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-GroupSnapshotsV319Test-Volume-617722805',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=a576e8d0-c0c6-43cc-a721-8d85e4b73ae8,host='ncdv-master-1@ceph#ceph',id=e60c704a-6696-43a8-b9ad-e88adb917173,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c71e0c400aa1407f81c69e5341fe1d89',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:15:55Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:15:55Z,use_quota=True,user_id='5ce94b54e7d943768a6e573b4ce758a6',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(9f140872-5f9f-4d35-80e6-10200deba263),volume_type_id=9f140872-5f9f-4d35-80e6-10200deba263)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:15:55.300933 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b515a64-8cec-425b-a158-de189f9a659c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:15:55.306580 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (2b515a64-8cec-425b-a158-de189f9a659c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:15:55.317553 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (617f6a97-b946-442a-820f-d3c98ca63ed9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:15:55.321569 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (617f6a97-b946-442a-820f-d3c98ca63ed9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'e60c704a-6696-43a8-b9ad-e88adb917173', 'volume_name': 'volume-e60c704a-6696-43a8-b9ad-e88adb917173', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:15:55.326299 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0bd3e9b7-21e5-4c14-8084-5748eb1f2fd6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:15:55.330908 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0bd3e9b7-21e5-4c14-8084-5748eb1f2fd6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:15:55.346975 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fcbbe7af-62d2-4a3a-95f6-ad77b0a5b87e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:15:55.348209 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Volume e60c704a-6696-43a8-b9ad-e88adb917173: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-e60c704a-6696-43a8-b9ad-e88adb917173', 'volume_size': 1}
Jul 02 21:15:55.348622 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] creating volume 'volume-e60c704a-6696-43a8-b9ad-e88adb917173' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:15:55.349333 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:15:55.507470 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] extra_specs: {} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:15:55.508099 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] extra_specs: {} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:15:55.512237 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (fcbbe7af-62d2-4a3a-95f6-ad77b0a5b87e) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-e60c704a-6696-43a8-b9ad-e88adb917173', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:15:55.518280 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0bbba949-35b4-4b86-b333-b21f73d13c1c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:15:55.532353 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Volume volume-e60c704a-6696-43a8-b9ad-e88adb917173 (e60c704a-6696-43a8-b9ad-e88adb917173): created successfully
Jul 02 21:15:55.537840 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (0bbba949-35b4-4b86-b333-b21f73d13c1c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:15:55.547359 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Flow 'volume_create_manager' (1d5db8f0-9186-4830-8c0e-28dcce364d0e) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:15:55.568083 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-147781c0-6417-41f6-9f20-129dca73c325 tempest-GroupSnapshotsV319Test-131159047 None] Created volume successfully.
Jul 02 21:15:56.745383 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-856a91d9-6e52-4ecc-8530-0c06d54b0515 tempest-GroupSnapshotsV319Test-131159047 None] GroupSnapshot 86061209-9f64-4172-b00e-a44f77bc08e5: creating.
Jul 02 21:15:56.874192 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-856a91d9-6e52-4ecc-8530-0c06d54b0515 tempest-GroupSnapshotsV319Test-131159047 None] Group snapshot 86061209-9f64-4172-b00e-a44f77bc08e5: creating. {{(pid=124013) create_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4081}}
Jul 02 21:15:56.925467 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-856a91d9-6e52-4ecc-8530-0c06d54b0515 tempest-GroupSnapshotsV319Test-131159047 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:15:57.569545 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-856a91d9-6e52-4ecc-8530-0c06d54b0515 tempest-GroupSnapshotsV319Test-131159047 None] group_snapshot 86061209-9f64-4172-b00e-a44f77bc08e5: created successfully
Jul 02 21:15:58.559819 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] group_snapshot 86061209-9f64-4172-b00e-a44f77bc08e5: deleting
Jul 02 21:15:58.628563 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] group_snapshot 86061209-9f64-4172-b00e-a44f77bc08e5: deleting {{(pid=124013) delete_group_snapshot /opt/stack/cinder/cinder/volume/manager.py:4256}}
Jul 02 21:15:58.678849 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:15:59.592598 ncdv-master-1 cinder-volume[124013]: WARNING cinder.quota [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] Deprecated: Default quota for resource: gigabytes_tempest-GroupSnapshotsV319Test-volume-type-986899837 is set by the default quota flag: quota_gigabytes_tempest-GroupSnapshotsV319Test-volume-type-986899837, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:15:59.615128 ncdv-master-1 cinder-volume[124013]: WARNING cinder.quota [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] Deprecated: Default quota for resource: snapshots_tempest-GroupSnapshotsV319Test-volume-type-986899837 is set by the default quota flag: quota_snapshots_tempest-GroupSnapshotsV319Test-volume-type-986899837, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:15:59.661092 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] Created reservations ['d897baa3-192e-4a00-a3cd-39827a479ad7', 'bf0aea24-2446-423c-87c7-fa81db35e1bc', '0a9164bb-d7c4-49f0-81d4-faab1a506f54', 'e71e22d2-f98a-4d29-b95e-c8778e8cae8a'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:15:59.762623 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-5cb3db98-1f1f-47ba-8bdf-666b0eb8f148 tempest-GroupSnapshotsV319Test-131159047 None] group_snapshot 86061209-9f64-4172-b00e-a44f77bc08e5: deleted successfully
Jul 02 21:16:00.458516 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:00.580928 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:16:00.581655 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] Volume volume-e60c704a-6696-43a8-b9ad-e88adb917173 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:16:00.610309 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] deleting rbd volume volume-e60c704a-6696-43a8-b9ad-e88adb917173 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:16:01.327792 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] Created reservations ['fb5865d3-5ee4-4401-bbe3-06f1cc95e42c'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:01.365717 ncdv-master-1 cinder-volume[124013]: WARNING cinder.quota [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] Deprecated: Default quota for resource: volumes_tempest-GroupSnapshotsV319Test-volume-type-986899837 is set by the default quota flag: quota_volumes_tempest-GroupSnapshotsV319Test-volume-type-986899837, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:16:01.413227 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] Created reservations ['2dafe042-70db-47c3-aacf-6d0ae21c1ed3', '0c92c3cc-9640-4e48-a2e9-33ba39b97180', 'ba0ade30-8302-4db9-abf2-32eefb1ad6b1', '84157b97-5da0-4ee7-ae4d-a162092bea22'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:01.656446 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:01.741889 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:01.801352 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:16:01.807979 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-fa590d99-4b85-4352-98fb-c4082938a49a tempest-GroupSnapshotsV319Test-131159047 None] Delete group completed successfully.
Jul 02 21:16:10.960868 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=62,cinder:UPDATE=21,cinder:DELETE=1,cinder:INSERT=5 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:16:10.970931 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:16:11.013193 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Flow 'volume_create_manager' (70cab23c-c301-4ae9-b68b-d29658bb11ff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:16:11.033372 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8492af8f-11da-461b-8452-4f3325b41b2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:11.082790 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (8492af8f-11da-461b-8452-4f3325b41b2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:16:11Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotsActionsTest-Volume-199635044',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=d1add273-3364-459b-94f9-06c214c9378e,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='48c2689f6b72433fa3a0cfd9a441481f',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:16:11Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:16:11Z,use_quota=True,user_id='ace2003567a44064bb848b39f6470cae',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:11.098681 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5be31236-9feb-4715-b211-e5b732001e95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:11.104028 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (5be31236-9feb-4715-b211-e5b732001e95) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:11.109722 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4cc34068-98e2-40ec-a66c-88ef2d217c05) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:11.115116 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (4cc34068-98e2-40ec-a66c-88ef2d217c05) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'd1add273-3364-459b-94f9-06c214c9378e', 'volume_name': 'volume-d1add273-3364-459b-94f9-06c214c9378e', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:11.121038 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (36140846-bb5c-443c-8726-1958abf0b90b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:11.125803 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (36140846-bb5c-443c-8726-1958abf0b90b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:11.147950 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0900ced4-12f6-4fa5-bbf0-03fbc5c85b2f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:11.149930 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Volume d1add273-3364-459b-94f9-06c214c9378e: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-d1add273-3364-459b-94f9-06c214c9378e', 'volume_size': 1}
Jul 02 21:16:11.153142 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] creating volume 'volume-d1add273-3364-459b-94f9-06c214c9378e' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:16:11.154199 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:11.299729 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:16:11.300629 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:16:11.305182 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (0900ced4-12f6-4fa5-bbf0-03fbc5c85b2f) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-d1add273-3364-459b-94f9-06c214c9378e', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:11.309936 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (160c1481-6e13-4f3a-979d-f15dae5c1482) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:11.338915 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Volume volume-d1add273-3364-459b-94f9-06c214c9378e (d1add273-3364-459b-94f9-06c214c9378e): created successfully
Jul 02 21:16:11.368833 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (160c1481-6e13-4f3a-979d-f15dae5c1482) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:11.387461 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Flow 'volume_create_manager' (70cab23c-c301-4ae9-b68b-d29658bb11ff) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:16:11.408325 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-a524e299-5e4c-4a6e-8250-8401ee39fdbe tempest-SnapshotsActionsTest-901202326 None] Created volume successfully.
Jul 02 21:16:12.344020 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-59c89d7c-6046-4a36-b7d0-cf65fe1c2b19 tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:12.699239 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-59c89d7c-6046-4a36-b7d0-cf65fe1c2b19 tempest-SnapshotsActionsTest-901202326 None] Create snapshot completed successfully
Jul 02 21:16:14.511536 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6023f775-966f-48be-acf8-9441be7455dd tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:15.690949 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-6023f775-966f-48be-acf8-9441be7455dd tempest-SnapshotsActionsTest-901202326 None] Create snapshot completed successfully
Jul 02 21:16:16.736421 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-d556afe1-15d1-444a-bda6-d1f336da8199 tempest-SnapshotsActionsTest-488595049 None] Acquiring lock "cinder-7e37caf5-498e-4be5-9b27-5151885ba1c1-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:16.754407 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-d556afe1-15d1-444a-bda6-d1f336da8199 tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-7e37caf5-498e-4be5-9b27-5151885ba1c1-delete_snapshot" acquired by "delete_snapshot" :: waited 0.022s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:16.759664 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-d556afe1-15d1-444a-bda6-d1f336da8199 tempest-SnapshotsActionsTest-488595049 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:17.735997 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=64,cinder:UPDATE=30,cinder:INSERT=4,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:16:17.775496 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-d556afe1-15d1-444a-bda6-d1f336da8199 tempest-SnapshotsActionsTest-488595049 None] Created reservations ['021d9216-81df-4b9b-a69b-82b1dfae335e', 'c4737a7d-56b4-48c1-a662-44d62d4c4cc6', 'c8e07586-955c-4fcd-aef8-54087fcf6ac4', '0d733abe-5b55-4c1b-a311-e59a1616ce1b'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:17.826171 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-d556afe1-15d1-444a-bda6-d1f336da8199 tempest-SnapshotsActionsTest-488595049 None] Delete snapshot completed successfully.
Jul 02 21:16:17.833585 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-d556afe1-15d1-444a-bda6-d1f336da8199 tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-7e37caf5-498e-4be5-9b27-5151885ba1c1-delete_snapshot" released by "delete_snapshot" :: held 1.079s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:18.206474 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-8e3620fc-044b-4731-b8e1-ed3872a5f2af tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:19.349850 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-8e3620fc-044b-4731-b8e1-ed3872a5f2af tempest-SnapshotsActionsTest-901202326 None] Create snapshot completed successfully
Jul 02 21:16:20.404242 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a643e2d8-ffc0-4f5e-800a-ab5fdd2b0357 tempest-SnapshotsActionsTest-488595049 None] Acquiring lock "cinder-256b53f3-fde0-418d-9806-c29ef835c224-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:20.416586 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a643e2d8-ffc0-4f5e-800a-ab5fdd2b0357 tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-256b53f3-fde0-418d-9806-c29ef835c224-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:20.418375 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a643e2d8-ffc0-4f5e-800a-ab5fdd2b0357 tempest-SnapshotsActionsTest-488595049 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:21.359214 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-a643e2d8-ffc0-4f5e-800a-ab5fdd2b0357 tempest-SnapshotsActionsTest-488595049 None] Created reservations ['65e198e7-e313-4072-9d3e-2d376ec7b3ee', '0ad72660-1723-4816-82cd-4f9f043daa80', '90f665b7-7c54-4c5c-a178-469f253b17c5', '67cd6ebe-ff83-48ba-aec8-649866b4d8ec'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:21.407038 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-a643e2d8-ffc0-4f5e-800a-ab5fdd2b0357 tempest-SnapshotsActionsTest-488595049 None] Delete snapshot completed successfully.
Jul 02 21:16:21.413717 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a643e2d8-ffc0-4f5e-800a-ab5fdd2b0357 tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-256b53f3-fde0-418d-9806-c29ef835c224-delete_snapshot" released by "delete_snapshot" :: held 0.997s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:21.774857 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-3a4d4efa-088a-40b4-a151-ef36e6f070fe tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:22.348309 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-3a4d4efa-088a-40b4-a151-ef36e6f070fe tempest-SnapshotsActionsTest-901202326 None] Create snapshot completed successfully
Jul 02 21:16:22.930691 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-c3395790-fcc6-4a09-9db1-675ce2f71183 tempest-SnapshotsActionsTest-488595049 None] Acquiring lock "cinder-d5a1d1fc-7055-45ef-8653-906191ee8530-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:22.941228 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-c3395790-fcc6-4a09-9db1-675ce2f71183 tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-d5a1d1fc-7055-45ef-8653-906191ee8530-delete_snapshot" acquired by "delete_snapshot" :: waited 0.010s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:22.943578 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-c3395790-fcc6-4a09-9db1-675ce2f71183 tempest-SnapshotsActionsTest-488595049 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:23.391603 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-c3395790-fcc6-4a09-9db1-675ce2f71183 tempest-SnapshotsActionsTest-488595049 None] Created reservations ['0a3b8dc8-78cf-4ca1-b34b-e939bdce8c0b', 'fc73074d-f9d9-4503-9081-73a15694a3ca', 'cf6b62f1-2a67-4b20-aff7-66a63680c639', '64b90976-eac4-48e0-9bba-e7395d2733d8'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:23.430713 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-c3395790-fcc6-4a09-9db1-675ce2f71183 tempest-SnapshotsActionsTest-488595049 None] Delete snapshot completed successfully.
Jul 02 21:16:23.435636 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-c3395790-fcc6-4a09-9db1-675ce2f71183 tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-d5a1d1fc-7055-45ef-8653-906191ee8530-delete_snapshot" released by "delete_snapshot" :: held 0.495s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:24.275492 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-c65f7a55-9071-4af1-a6b6-2fd262dac04a tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:25.403919 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-c65f7a55-9071-4af1-a6b6-2fd262dac04a tempest-SnapshotsActionsTest-901202326 None] Create snapshot completed successfully
Jul 02 21:16:26.470584 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a2843cb0-f05e-4683-a693-d6c64b66400e tempest-SnapshotsActionsTest-488595049 None] Acquiring lock "cinder-7e313870-2715-4fc7-8098-24764c3796d6-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:26.493420 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a2843cb0-f05e-4683-a693-d6c64b66400e tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-7e313870-2715-4fc7-8098-24764c3796d6-delete_snapshot" acquired by "delete_snapshot" :: waited 0.023s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:26.494558 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a2843cb0-f05e-4683-a693-d6c64b66400e tempest-SnapshotsActionsTest-488595049 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:27.453529 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-a2843cb0-f05e-4683-a693-d6c64b66400e tempest-SnapshotsActionsTest-488595049 None] Created reservations ['14700779-6b12-4bf5-8c4d-bc935766a5c3', 'c6b68272-c8ab-4e7f-9df7-14ba348d4fd0', 'e3c0882e-0bca-439f-8270-779d49f8af3a', 'b518df48-ff25-41cb-b520-ae6189f7bd43'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:27.488945 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=62,cinder:UPDATE=25,cinder:DELETE=2,cinder:INSERT=11 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:16:27.504003 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-a2843cb0-f05e-4683-a693-d6c64b66400e tempest-SnapshotsActionsTest-488595049 None] Delete snapshot completed successfully.
Jul 02 21:16:27.511692 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a2843cb0-f05e-4683-a693-d6c64b66400e tempest-SnapshotsActionsTest-488595049 None] Lock "cinder-7e313870-2715-4fc7-8098-24764c3796d6-delete_snapshot" released by "delete_snapshot" :: held 1.019s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:28.046620 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-d465277f-edc9-4f5d-a4b6-76220139bc16 tempest-SnapshotsActionsTest-901202326 None] Acquiring lock "cinder-30f5dcb7-7088-4b5a-9233-8ada44dfce5c-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:28.072767 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-d465277f-edc9-4f5d-a4b6-76220139bc16 tempest-SnapshotsActionsTest-901202326 None] Lock "cinder-30f5dcb7-7088-4b5a-9233-8ada44dfce5c-delete_snapshot" acquired by "delete_snapshot" :: waited 0.026s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:28.075653 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-d465277f-edc9-4f5d-a4b6-76220139bc16 tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:29.521083 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-d465277f-edc9-4f5d-a4b6-76220139bc16 tempest-SnapshotsActionsTest-901202326 None] Created reservations ['a8709833-eab4-4749-887c-2ad951d6e173', 'dc46618f-14e0-4895-9acd-ed19dbb31186', 'cc314a07-0361-42e3-a95b-3724b1f4215d', '571aa24b-8f64-4fe7-a3eb-676ccb06332c'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:29.559429 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:UPDATE=32,cinder:SELECT=59,cinder:DELETE=2,cinder:INSERT=7 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:16:29.575406 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-d465277f-edc9-4f5d-a4b6-76220139bc16 tempest-SnapshotsActionsTest-901202326 None] Delete snapshot completed successfully.
Jul 02 21:16:29.582500 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-d465277f-edc9-4f5d-a4b6-76220139bc16 tempest-SnapshotsActionsTest-901202326 None] Lock "cinder-30f5dcb7-7088-4b5a-9233-8ada44dfce5c-delete_snapshot" released by "delete_snapshot" :: held 1.510s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:30.287794 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Acquiring lock "cinder-d1add273-3364-459b-94f9-06c214c9378e-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:30.322098 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Lock "cinder-d1add273-3364-459b-94f9-06c214c9378e-delete_volume" acquired by "delete_volume" :: waited 0.034s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:30.433198 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:30.534180 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:16:30.534873 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Volume volume-d1add273-3364-459b-94f9-06c214c9378e is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:16:30.548966 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] deleting rbd volume volume-d1add273-3364-459b-94f9-06c214c9378e {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:16:31.266692 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Created reservations ['3066d6e2-32dd-4585-8f40-92e03f1c0ae6', '6493173b-dee4-4ae6-95d3-8e5def718565', 'e6e914bc-b225-4b9a-aa97-eece1b7543a8', 'd47a4ffc-3359-48cf-81e1-c57325c79fc0'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:31.332477 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:31.408363 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:31.473302 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:16:31.478447 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Deleted volume successfully.
Jul 02 21:16:31.510903 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dead3469-533b-45e6-98b3-40df7ac0ba18 tempest-SnapshotsActionsTest-901202326 None] Lock "cinder-d1add273-3364-459b-94f9-06c214c9378e-delete_volume" released by "delete_volume" :: held 1.189s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:34.165894 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:16:34.168810 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:34.273040 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:34.342401 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:16:34.434587 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [req-6d1f0880-8b55-4777-8cde-7419f44f43f5 req-9205db10-d8c8-4116-b94c-b1e04e1270be tempest-AttachVolumeShelveTestJSON-1224937377 None] Terminate volume connection completed successfully.
Jul 02 21:16:34.435419 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [req-6d1f0880-8b55-4777-8cde-7419f44f43f5 req-9205db10-d8c8-4116-b94c-b1e04e1270be tempest-AttachVolumeShelveTestJSON-1224937377 None] Deleting attachment ebfbfdcc-832d-4238-bc57-3657bca2455f. {{(pid=124013) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:16:44.215606 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:16:44.242783 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Flow 'volume_create_manager' (4189127d-96b6-4b9b-aa38-e029de5254a9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:16:44.249761 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (738964c0-660a-4f31-8fb4-7586fb08740c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:44.274008 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (738964c0-660a-4f31-8fb4-7586fb08740c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:16:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-SnapshotMetadataTestJSON-Volume-654159652',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=6dea1183-a9a8-4cb7-a679-f2c8f89eeca9,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f2247d263b2f463e96c03a04d1405210',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:16:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:16:44Z,use_quota=True,user_id='02e57fa01d2a4e7a8e6b398aabf2f453',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:44.281942 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (23f1cfcb-6483-44ea-ba61-d1613d55bac7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:44.287143 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (23f1cfcb-6483-44ea-ba61-d1613d55bac7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:44.293538 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (614ac65f-dae0-406c-9893-971090868092) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:44.299008 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (614ac65f-dae0-406c-9893-971090868092) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '6dea1183-a9a8-4cb7-a679-f2c8f89eeca9', 'volume_name': 'volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:44.305129 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8e645063-7abf-4208-a0b2-1818b5735d2e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:44.314115 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (8e645063-7abf-4208-a0b2-1818b5735d2e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:44.319935 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (72dcbc89-8513-4100-a256-54b8491224e4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:44.321367 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Volume 6dea1183-a9a8-4cb7-a679-f2c8f89eeca9: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9', 'volume_size': 1}
Jul 02 21:16:44.321766 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] creating volume 'volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:16:44.322543 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:44.433591 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:16:44.434189 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:16:44.441303 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (72dcbc89-8513-4100-a256-54b8491224e4) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:44.451956 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e706165f-ed6c-431c-b568-1797b57e2905) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:44.473411 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Volume volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9 (6dea1183-a9a8-4cb7-a679-f2c8f89eeca9): created successfully
Jul 02 21:16:44.504298 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (e706165f-ed6c-431c-b568-1797b57e2905) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:44.511820 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Flow 'volume_create_manager' (4189127d-96b6-4b9b-aa38-e029de5254a9) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:16:44.534639 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-82a64514-8f13-4b5e-a530-645f18f3798c tempest-SnapshotMetadataTestJSON-1049041747 None] Created volume successfully.
Jul 02 21:16:45.816756 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-63365e6d-8b22-4be6-9452-e6fd31aa92f9 tempest-SnapshotMetadataTestJSON-1049041747 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:46.552910 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-63365e6d-8b22-4be6-9452-e6fd31aa92f9 tempest-SnapshotMetadataTestJSON-1049041747 None] Create snapshot completed successfully
Jul 02 21:16:48.150277 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-72e982be-913a-4eee-be08-4886f559c574 tempest-SnapshotMetadataTestJSON-1049041747 None] Acquiring lock "cinder-b0ea4a06-955a-45f6-9659-f79437a126af-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:48.165608 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-72e982be-913a-4eee-be08-4886f559c574 tempest-SnapshotMetadataTestJSON-1049041747 None] Lock "cinder-b0ea4a06-955a-45f6-9659-f79437a126af-delete_snapshot" acquired by "delete_snapshot" :: waited 0.011s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:48.170312 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-72e982be-913a-4eee-be08-4886f559c574 tempest-SnapshotMetadataTestJSON-1049041747 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:49.467597 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-72e982be-913a-4eee-be08-4886f559c574 tempest-SnapshotMetadataTestJSON-1049041747 None] Created reservations ['0537c293-271f-40ce-b759-9d6ecb76318a', 'b2b9c217-8a44-4093-94b4-e3cc81272a24', '6c911bf2-0add-4239-9b2f-5bc5a0086cfa', 'b24d6b3d-fe25-4976-a464-40c2764a5b44'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:49.524610 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-72e982be-913a-4eee-be08-4886f559c574 tempest-SnapshotMetadataTestJSON-1049041747 None] Delete snapshot completed successfully.
Jul 02 21:16:49.535443 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-72e982be-913a-4eee-be08-4886f559c574 tempest-SnapshotMetadataTestJSON-1049041747 None] Lock "cinder-b0ea4a06-955a-45f6-9659-f79437a126af-delete_snapshot" released by "delete_snapshot" :: held 1.374s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:50.423967 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Acquiring lock "cinder-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:16:50.465485 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Lock "cinder-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9-delete_volume" acquired by "delete_volume" :: waited 0.042s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:16:50.560082 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:50.683423 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:16:50.684497 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Volume volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:16:50.705256 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] deleting rbd volume volume-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:16:51.210085 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Created reservations ['5ebf97cc-fbf3-4bb1-a1eb-c58860d294c3', '2195f648-897a-49e2-a7f9-03299bf13f1e', 'c90bb272-1c41-4dbb-b90a-57c5d2dd2534', '94e527cc-b795-4bc6-9b73-e16ecee0db3e'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:16:51.357845 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:51.445547 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:51.523623 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:16:51.528471 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Deleted volume successfully.
Jul 02 21:16:51.556132 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-dddd05eb-781c-4033-940c-15da505c8cee tempest-SnapshotMetadataTestJSON-1049041747 None] Lock "cinder-6dea1183-a9a8-4cb7-a679-f2c8f89eeca9-delete_volume" released by "delete_volume" :: held 1.091s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:16:58.004000 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=57,cinder:INSERT=6,cinder:UPDATE=33,cinder:DELETE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:16:58.025738 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:16:58.126436 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Flow 'volume_create_manager' (89954f3a-c993-4af8-9fe4-69f1067e2a73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:16:58.152405 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e5e2d964-cae1-49d7-8e68-5df44b3f29b7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:58.208262 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (e5e2d964-cae1-49d7-8e68-5df44b3f29b7) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:16:57Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesDeleteCascade-Volume-1398516468',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=a6009591-4660-44c9-8b6a-cb746a993532,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='7b769d890fea450699957ed0aa6bd804',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:16:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:16:58Z,use_quota=True,user_id='466dc5d829ac45389fead7c50b68e608',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:58.222741 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e8a69398-e248-4b81-ad58-f8a2274e509a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:58.234631 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (e8a69398-e248-4b81-ad58-f8a2274e509a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:58.239621 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9694da33-972c-4525-98f2-6401f92bdeff) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:58.264857 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (9694da33-972c-4525-98f2-6401f92bdeff) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a6009591-4660-44c9-8b6a-cb746a993532', 'volume_name': 'volume-a6009591-4660-44c9-8b6a-cb746a993532', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:58.273657 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ebca10f8-8710-43d0-92e8-2c61bd679d1b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:58.283634 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (ebca10f8-8710-43d0-92e8-2c61bd679d1b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:58.291933 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c22d36e9-89a9-4f55-824f-48eb87ea22ba) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:58.296964 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Volume a6009591-4660-44c9-8b6a-cb746a993532: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a6009591-4660-44c9-8b6a-cb746a993532', 'volume_size': 1}
Jul 02 21:16:58.300793 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] creating volume 'volume-a6009591-4660-44c9-8b6a-cb746a993532' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:16:58.301704 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:16:58.474890 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:16:58.475801 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:16:58.481076 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c22d36e9-89a9-4f55-824f-48eb87ea22ba) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a6009591-4660-44c9-8b6a-cb746a993532', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:58.511242 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f25b0685-6181-46f2-afc6-bec6037556c2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:16:58.546927 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Volume volume-a6009591-4660-44c9-8b6a-cb746a993532 (a6009591-4660-44c9-8b6a-cb746a993532): created successfully
Jul 02 21:16:58.559188 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (f25b0685-6181-46f2-afc6-bec6037556c2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:16:58.571054 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Flow 'volume_create_manager' (89954f3a-c993-4af8-9fe4-69f1067e2a73) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:16:58.609588 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-fe17965e-1a64-400d-b429-2684e23e44fc tempest-VolumesDeleteCascade-100262499 None] Created volume successfully.
Jul 02 21:16:59.403517 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-af818547-76dc-4069-8200-bc0397f180dc tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:00.756011 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=58,cinder:INSERT=8,cinder:UPDATE=26,cinder:DELETE=2 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:17:00.787136 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-af818547-76dc-4069-8200-bc0397f180dc tempest-VolumesDeleteCascade-100262499 None] Create snapshot completed successfully
Jul 02 21:17:01.991196 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-344035a7-c798-46cf-90bc-1018e8225cf9 tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:02.875005 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-344035a7-c798-46cf-90bc-1018e8225cf9 tempest-VolumesDeleteCascade-100262499 None] Create snapshot completed successfully
Jul 02 21:17:03.231016 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Acquiring lock "cinder-a6009591-4660-44c9-8b6a-cb746a993532-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:17:03.264807 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Lock "cinder-a6009591-4660-44c9-8b6a-cb746a993532-delete_volume" acquired by "delete_volume" :: waited 0.031s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:17:03.363802 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Performing cascade delete. {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/manager.py:968}}
Jul 02 21:17:03.391639 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Acquiring lock "cinder-7049343b-d862-44cb-b4e6-41cb758eb062-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:17:03.410937 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Lock "cinder-7049343b-d862-44cb-b4e6-41cb758eb062-delete_snapshot" acquired by "delete_snapshot" :: waited 0.019s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:17:03.446115 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:04.813494 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Created reservations ['3c68bfc1-d574-47ff-8910-51448bae54c2', '7027c58a-9026-437c-9b93-037be0053a60', 'da7d6dc3-8b4b-4cd0-9991-f5c4c022b222', 'bb5db1a5-b170-4bb8-b741-8e75ece48d05'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:17:04.888182 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Delete snapshot completed successfully.
Jul 02 21:17:04.895183 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Lock "cinder-7049343b-d862-44cb-b4e6-41cb758eb062-delete_snapshot" released by "delete_snapshot" :: held 1.484s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:17:04.897704 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Acquiring lock "cinder-d5bab60c-531d-4867-a56d-8a7e33f6f887-delete_snapshot" by "delete_snapshot" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:17:04.921544 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Lock "cinder-d5bab60c-531d-4867-a56d-8a7e33f6f887-delete_snapshot" acquired by "delete_snapshot" :: waited 0.025s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:17:04.947885 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:05.834223 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Created reservations ['f05a6307-afc3-4ffa-9f97-f7c2f478a869', 'b13545af-ef8a-4588-ab8a-b7659628d7a5', 'dd72dd62-3a7f-4aef-a088-a372dfec4e0c', 'a955ea82-c9a4-42a4-a2f8-98d3f8a059a7'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:17:05.971560 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Delete snapshot completed successfully.
Jul 02 21:17:05.983697 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Lock "cinder-d5bab60c-531d-4867-a56d-8a7e33f6f887-delete_snapshot" released by "delete_snapshot" :: held 1.061s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:17:05.986569 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Snapshots deleted, issuing volume delete {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/manager.py:983}}
Jul 02 21:17:05.988701 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:06.118937 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:17:06.119830 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Volume volume-a6009591-4660-44c9-8b6a-cb746a993532 is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:17:06.158978 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] deleting rbd volume volume-a6009591-4660-44c9-8b6a-cb746a993532 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:17:07.188237 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Created reservations ['4bee3b12-530e-4548-883a-6eab9b2a6832', 'd239d284-56b4-4f5d-9f50-2ad3f260f596', '6b6d3a31-8e90-4fe7-9cb5-363c361d1bb0', '3e3a8bad-6fa9-473c-91a6-648161eb1424'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:17:07.279179 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:07.410271 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:07.521966 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:17:07.527261 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Deleted volume successfully.
Jul 02 21:17:07.554892 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-6c95d36d-07c1-43f7-850a-a7720232abbf tempest-VolumesDeleteCascade-100262499 None] Lock "cinder-a6009591-4660-44c9-8b6a-cb746a993532-delete_volume" released by "delete_volume" :: held 4.292s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:17:15.104820 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:17:15.142005 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Flow 'volume_create_manager' (65f8c733-5ba4-46d2-b1a0-009365a179e6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:17:15.150882 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fd3a130f-41f4-46d4-884d-e7482ea5e68b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:15.177367 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (fd3a130f-41f4-46d4-884d-e7482ea5e68b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:17:14Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Volume-325361192',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=4dbabada-9dc6-484e-8805-cd840a33009a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e2b1245038184e93b3e6893d777b5ef1',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:17:15Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:17:15Z,use_quota=True,user_id='529e1697898a4a91a39c8fa841462d8f',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:15.184503 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b752bdcc-8bc2-40eb-abc5-17d08b9a0bf5) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:15.193750 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (b752bdcc-8bc2-40eb-abc5-17d08b9a0bf5) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:15.199865 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d4f211cc-3856-4124-8ad0-5fcce34c0f81) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:15.205276 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d4f211cc-3856-4124-8ad0-5fcce34c0f81) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '4dbabada-9dc6-484e-8805-cd840a33009a', 'volume_name': 'volume-4dbabada-9dc6-484e-8805-cd840a33009a', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:15.212095 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dc0672f8-db94-4c49-86b1-b8b4c9ce1543) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:15.217190 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (dc0672f8-db94-4c49-86b1-b8b4c9ce1543) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:15.223231 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3fbc42fa-fdbc-4da9-9e3d-74a9e29e0ddf) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:15.225237 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Volume 4dbabada-9dc6-484e-8805-cd840a33009a: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-4dbabada-9dc6-484e-8805-cd840a33009a', 'volume_size': 1}
Jul 02 21:17:15.225942 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] creating volume 'volume-4dbabada-9dc6-484e-8805-cd840a33009a' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:17:15.226960 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:15.463280 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:17:15.465110 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:17:15.471365 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (3fbc42fa-fdbc-4da9-9e3d-74a9e29e0ddf) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-4dbabada-9dc6-484e-8805-cd840a33009a', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:15.477628 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d5853454-48c9-495f-9b1a-c2ea3599ee00) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:15.498964 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Volume volume-4dbabada-9dc6-484e-8805-cd840a33009a (4dbabada-9dc6-484e-8805-cd840a33009a): created successfully
Jul 02 21:17:15.516828 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d5853454-48c9-495f-9b1a-c2ea3599ee00) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:15.526903 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Flow 'volume_create_manager' (65f8c733-5ba4-46d2-b1a0-009365a179e6) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:17:15.549510 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-076c726f-ad46-4058-8254-875011967894 tempest-VolumesBackupsV39Test-89977103 None] Created volume successfully.
Jul 02 21:17:16.590157 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-07-02T21:17:16Z,data_timestamp=2022-07-02T21:17:16Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesBackupsV39Test-Backup-1279083432',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=ca288613-3ffa-480e-9bbc-3a8e790f5cd2,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='e2b1245038184e93b3e6893d777b5ef1',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id=None,status='creating',temp_snapshot_id=None,temp_volume_id=None,updated_at=None,user_id='529e1697898a4a91a39c8fa841462d8f',volume_id=4dbabada-9dc6-484e-8805-cd840a33009a)
Jul 02 21:17:16.601885 ncdv-master-1 cinder-volume[124013]: INFO cinder.rpc [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Automatically selected cinder-backup objects version 1.39 as minimum service version.
Jul 02 21:17:16.619629 ncdv-master-1 cinder-volume[124013]: INFO cinder.rpc [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Automatically selected cinder-backup RPC version 2.3 as minimum service version.
Jul 02 21:17:16.623821 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.backup.rpcapi [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] continue_backup in rpcapi backup_id ca288613-3ffa-480e-9bbc-3a8e790f5cd2 {{(pid=124013) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}}
Jul 02 21:17:25.928489 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:17:27.216331 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.287s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:17:27.217509 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-4dbabada-9dc6-484e-8805-cd840a33009a', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '4dbabada-9dc6-484e-8805-cd840a33009a', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:17:27.244961 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Initialize volume connection completed successfully.
Jul 02 21:17:29.960697 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:UPDATE=27,cinder:SELECT=53,cinder:DELETE=1,cinder:INSERT=8 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:17:31.233114 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Terminate volume connection completed successfully.
Jul 02 21:17:31.256163 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-1d65037f-6a35-4c7f-8923-104850b71c26 tempest-VolumesBackupsV39Test-89977103 None] Remove volume export completed successfully.
Jul 02 21:17:33.856569 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Acquiring lock "cinder-4dbabada-9dc6-484e-8805-cd840a33009a-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:17:33.868930 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Lock "cinder-4dbabada-9dc6-484e-8805-cd840a33009a-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:17:33.883087 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=60,cinder:UPDATE=23,cinder:DELETE=3,cinder:INSERT=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:17:33.929393 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:35.078600 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Volume volume-4dbabada-9dc6-484e-8805-cd840a33009a is not a clone. {{(pid=124013) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:17:35.128743 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] deleting rbd volume volume-4dbabada-9dc6-484e-8805-cd840a33009a {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:17:35.137345 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:17:35.139978 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:35.256235 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:35.337935 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:17:35.707298 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.quota [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Created reservations ['ad24f00e-f117-4454-b1c8-2b713c40db71', '23991761-69ec-4312-b8d6-6265488e2a1c', '80f801f4-3ac8-424f-81ed-49dd02558db9', '9413ee2b-48e5-4db4-b29a-b6e64b43786b'] {{(pid=124013) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:17:35.766229 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:35.823927 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:35.896955 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:17:35.901152 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Deleted volume successfully.
Jul 02 21:17:35.922172 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-74819248-bd21-4be1-95ef-7244d2376a20 tempest-VolumesBackupsV39Test-89977103 None] Lock "cinder-4dbabada-9dc6-484e-8805-cd840a33009a-delete_volume" released by "delete_volume" :: held 2.053s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:17:44.071141 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:17:44.132470 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (7ce1b961-83c6-41ec-a87d-6c3f5e0076c3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:17:44.141266 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dc5e6cf6-db38-4cca-9458-e0fff68cc46a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:44.230056 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (dc5e6cf6-db38-4cca-9458-e0fff68cc46a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:17:44Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-690222677',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=792de71d-1208-4d3b-90d3-efdd17ab60ab,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c7534889ba5b48338c27c0556be3c6f0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:17:44Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:17:44Z,use_quota=True,user_id='2016769bf3a34d22992b45f9974b99a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:44.246420 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1809bf9c-6aa4-4425-b1be-50c0d0a8282b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:44.255280 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1809bf9c-6aa4-4425-b1be-50c0d0a8282b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:44.294430 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c8062367-a3f2-4efa-bd4a-2fe4ceef5ef9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:44.308787 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (c8062367-a3f2-4efa-bd4a-2fe4ceef5ef9) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '792de71d-1208-4d3b-90d3-efdd17ab60ab', 'volume_name': 'volume-792de71d-1208-4d3b-90d3-efdd17ab60ab', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:44.322606 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5281b398-8050-4007-a9e8-cc62f8d84f8a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:44.331399 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (5281b398-8050-4007-a9e8-cc62f8d84f8a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:44.352013 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (eb3579b0-81a3-44c0-9342-533ccccc7bf6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:44.353561 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume 792de71d-1208-4d3b-90d3-efdd17ab60ab: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-792de71d-1208-4d3b-90d3-efdd17ab60ab', 'volume_size': 1}
Jul 02 21:17:44.354266 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] creating volume 'volume-792de71d-1208-4d3b-90d3-efdd17ab60ab' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:17:44.355283 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:44.589206 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:17:44.589612 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:17:44.593774 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (eb3579b0-81a3-44c0-9342-533ccccc7bf6) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-792de71d-1208-4d3b-90d3-efdd17ab60ab', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:44.614127 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a8d36a65-b7b9-4a4a-99ef-dc736deb47a4) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:44.645247 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-792de71d-1208-4d3b-90d3-efdd17ab60ab (792de71d-1208-4d3b-90d3-efdd17ab60ab): created successfully
Jul 02 21:17:44.650109 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (a8d36a65-b7b9-4a4a-99ef-dc736deb47a4) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:44.658546 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (7ce1b961-83c6-41ec-a87d-6c3f5e0076c3) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:17:44.695881 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-6b11e987-dec0-4ae6-a4a2-d263f8cb0295 tempest-VolumesSnapshotTestJSON-1740353321 None] Created volume successfully.
Jul 02 21:17:45.619273 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-b6305f30-fdee-4864-ac91-33d9e645a472 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:46.868027 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-b6305f30-fdee-4864-ac91-33d9e645a472 tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:17:48.052246 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.driver [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Creating a new backup for volume volume-792de71d-1208-4d3b-90d3-efdd17ab60ab. {{(pid=124013) _get_backup_volume_temp_volume /opt/stack/cinder/cinder/volume/driver.py:1213}}
Jul 02 21:17:48.116635 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] cloning volumes/volume-792de71d-1208-4d3b-90d3-efdd17ab60ab@snapshot-7f1b0b27-ec0f-4fdc-ae9c-b9eb4c143e6d to volume-e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90 {{(pid=124013) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}}
Jul 02 21:17:48.118414 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:48.202572 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:48.436168 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:17:48.436762 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:17:48.442023 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:48.504706 ncdv-master-1 cinder-volume[124013]: WARNING cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues
Jul 02 21:17:48.551607 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Calling backup continue_backup for: Backup(availability_zone=None,container=None,created_at=2022-07-02T21:17:47Z,data_timestamp=2022-07-02T21:17:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Backup-1171562765',encryption_key_id=None,fail_reason=None,host='ncdv-master-1',id=285a5599-efb0-48b4-b0d4-306b249484ec,metadata={},num_dependent_backups=0,object_count=0,parent=None,parent_id=None,project_id='c7534889ba5b48338c27c0556be3c6f0',restore_volume_id=None,service='cinder.backup.drivers.ceph.CephBackupDriver',service_metadata=None,size=1,snapshot_id='7f1b0b27-ec0f-4fdc-ae9c-b9eb4c143e6d',status='creating',temp_snapshot_id=None,temp_volume_id='e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90',updated_at=None,user_id='2016769bf3a34d22992b45f9974b99a2',volume_id=792de71d-1208-4d3b-90d3-efdd17ab60ab)
Jul 02 21:17:48.553273 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.backup.rpcapi [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] continue_backup in rpcapi backup_id 285a5599-efb0-48b4-b0d4-306b249484ec {{(pid=124013) continue_backup /opt/stack/cinder/cinder/backup/rpcapi.py:64}}
Jul 02 21:17:48.852077 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:17:49.984356 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_concurrency.processutils [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.131s {{(pid=124013) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:17:49.984356 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': 'e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90', 'discard': True}} {{(pid=124013) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:17:50.014781 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Initialize volume connection completed successfully.
Jul 02 21:17:58.460523 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:17:58.510012 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Flow 'volume_create_manager' (68971ba5-630e-435e-8e6a-b30931d830f7) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:17:58.525572 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (26581cac-230d-47b5-8e43-e44a9fdb408d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:58.569868 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (26581cac-230d-47b5-8e43-e44a9fdb408d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:17:58Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-AttachVolumeShelveTestJSON-volume-457704604',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=eb6fa52b-f4db-4f5b-80e7-c80177b5e350,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='f644f2d1c56c43cf851deb020af7161e',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:17:58Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:17:58Z,use_quota=True,user_id='984631f5a77a48c5a9619e33e608dd59',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:58.586806 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f649eea0-602a-41e1-b017-64f6f960de32) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:58.592209 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (f649eea0-602a-41e1-b017-64f6f960de32) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:58.601644 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0b6954a3-defc-407d-8149-6315b8f47883) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:58.605841 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (0b6954a3-defc-407d-8149-6315b8f47883) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'eb6fa52b-f4db-4f5b-80e7-c80177b5e350', 'volume_name': 'volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:58.629574 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b71ec7ae-a472-4bcb-a7f7-f6faee1d5bed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:58.633915 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (b71ec7ae-a472-4bcb-a7f7-f6faee1d5bed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:58.638549 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e6fc2792-acd8-44b0-be72-08ab0d1fd441) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:58.649209 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume eb6fa52b-f4db-4f5b-80e7-c80177b5e350: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350', 'volume_size': 1}
Jul 02 21:17:58.650336 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] creating volume 'volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350' {{(pid=124013) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:17:58.651828 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:17:59.183056 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:17:59.184443 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=124013) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:17:59.189911 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (e6fc2792-acd8-44b0-be72-08ab0d1fd441) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350', 'volume_size': 1}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:59.207318 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (52396f66-b7bd-4d29-8500-b1daaaa849a8) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:17:59.253106 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350 (eb6fa52b-f4db-4f5b-80e7-c80177b5e350): created successfully
Jul 02 21:17:59.258405 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (52396f66-b7bd-4d29-8500-b1daaaa849a8) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:17:59.275048 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Flow 'volume_create_manager' (68971ba5-630e-435e-8e6a-b30931d830f7) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:17:59.299249 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-c5d77cfc-f0cb-41d6-a52b-a50b3e5ddcd5 tempest-AttachVolumeShelveTestJSON-1224937377 None] Created volume successfully.
Jul 02 21:18:09.970384 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=38,cinder:INSERT=3,cinder:UPDATE=12,cinder:DELETE=3 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:18:09.971888 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=40,cinder:UPDATE=17,cinder:INSERT=3,cinder:DELETE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:18:10.221858 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Terminate volume connection completed successfully.
Jul 02 21:18:10.289687 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Remove volume export completed successfully.
Jul 02 21:18:10.354415 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90-delete_volume" by "delete_volume" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:18:10.376109 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90-delete_volume" acquired by "delete_volume" :: waited 0.022s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:18:10.410192 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:10.596684 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=124013) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:18:10.636899 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90 {{(pid=124013) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:18:11.345170 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.manager [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Deleted volume successfully.
Jul 02 21:18:11.351698 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-7963ac1e-fd5e-435e-895d-e634440f25d6 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-e6e0eabc-7407-4471-b7e2-2a8d8e7d2b90-delete_volume" released by "delete_volume" :: held 0.976s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:18:21.337957 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=2,cinder:UPDATE=6 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:18:35.139225 ncdv-master-1 cinder-volume[124013]: DEBUG oslo_service.periodic_task [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=124013) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:18:35.140203 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:35.232009 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:35.334744 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.manager [None req-317ab14d-3173-4b1f-bf83-d03dc318848f None None] Notifying Schedulers of capabilities ... {{(pid=124013) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:18:39.975951 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:UPDATE=1 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:18:45.221124 ncdv-master-1 cinder-volume[124013]: DEBUG dbcounter [-] [124013] Writing DB stats cinder:SELECT=16,cinder:UPDATE=4 {{(pid=124013) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:18:45.245653 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=124013) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:18:45.267683 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (a282599c-41a1-42c2-9d3f-177970c4fe97) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:18:45.275279 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19aed9aa-393d-422b-875c-daf89d85f090) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:18:45.294631 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (19aed9aa-393d-422b-875c-daf89d85f090) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:18:45Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-826395167',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=56999857-0213-428a-a613-de4e9f99f593,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c7534889ba5b48338c27c0556be3c6f0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:18:45Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:18:45Z,use_quota=True,user_id='2016769bf3a34d22992b45f9974b99a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:18:45.301858 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (024775b2-f5af-422b-954b-4b3af61e176b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:18:45.306905 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (024775b2-f5af-422b-954b-4b3af61e176b) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:18:45.312872 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb6d3756-e685-4058-ad94-36b1052c9a88) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:18:45.457518 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (cb6d3756-e685-4058-ad94-36b1052c9a88) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '56999857-0213-428a-a613-de4e9f99f593', 'volume_name': 'volume-56999857-0213-428a-a613-de4e9f99f593', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072a69820>}' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:18:45.465634 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ff79dc9-01db-40dc-9834-e3849d83dd4f) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:18:45.475066 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (4ff79dc9-01db-40dc-9834-e3849d83dd4f) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:18:45.494229 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.manager [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (920f3e25-5850-4e4a-844d-b622a9dc7669) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=124013) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:18:45.494229 ncdv-master-1 cinder-volume[124013]: INFO cinder.volume.flows.manager.create_volume [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume 56999857-0213-428a-a613-de4e9f99f593: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-56999857-0213-428a-a613-de4e9f99f593', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f8072a69820>}
Jul 02 21:18:45.494633 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Cloning 56999857-0213-428a-a613-de4e9f99f593 from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=124013) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:18:45.494633 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:18:45.506394 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.014s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:18:45.537885 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 18, 45, 521687)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:18:45.538519 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:18:45.539062 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Found cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}}
Jul 02 21:18:45.544420 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.coordination [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 0.038s {{(pid=124013) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:18:45.545748 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:18:45.560017 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 18, 45, 553392)}. {{(pid=124013) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:18:45.562862 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.image.cache [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=124013) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:18:45.563287 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.flows.manager.create_volume [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] Creating from source image-volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45 {{(pid=124013) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}}
Jul 02 21:18:45.594506 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=124013) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:45.733481 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] creating snapshot='volume-56999857-0213-428a-a613-de4e9f99f593.clone_snap' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:18:46.247587 ncdv-master-1 cinder-volume[124013]: DEBUG cinder.volume.drivers.rbd [None req-a97ab6b5-00bb-4019-a752-1774d0f02a13 tempest-VolumesSnapshotTestJSON-1740353321 None] cloning 'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45@volume-56999857-0213-428a-a613-de4e9f99f593.clone_snap' to 'volume-56999857-0213-428a-a613-de4e9f99f593' {{(pid=124013) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:785}}
Jul 02 21:18:46.530793 ncdv-master-1 cinder-volume[124013]: terminate called after throwing an instance of 'std::system_error'
Jul 02 21:18:46.530918 ncdv-master-1 cinder-volume[124013]:   what():  Invalid argument
Jul 02 21:18:46.555046 ncdv-master-1 cinder-volume[123840]: INFO oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Child 124013 killed by signal 6
Jul 02 21:18:46.564246 ncdv-master-1 cinder-volume[123840]: DEBUG oslo_service.service [None req-bbcc356b-83e0-4b4d-a78c-080bed19fed8 None None] Started child 156391 {{(pid=123840) _start_child /usr/local/lib/python3.8/dist-packages/oslo_service/service.py:575}}
Jul 02 21:18:46.581412 ncdv-master-1 cinder-volume[156391]: INFO cinder.service [-] Starting cinder-volume node (version 20.1.0)
Jul 02 21:18:46.680260 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Starting volume driver RBDDriver (1.2.0)
Jul 02 21:18:46.685066 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:46.765618 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:46.813119 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:46.871120 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.driver [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Driver hasn't implemented _init_vendor_properties()
Jul 02 21:18:46.871953 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.driver [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Initialized capabilities list: {'vendor_name': 'Open Source', 'driver_version': '1.2.0', 'storage_protocol': 'ceph', 'total_capacity_gb': 19.83, 'free_capacity_gb': 19.28, 'reserved_percentage': 0, 'multiattach': True, 'thin_provisioning_support': True, 'max_over_subscription_ratio': '20.0', 'location_info': 'ceph:/etc/ceph/ceph.conf:2b29f0f5-9ab4-46b2-a4f8-882559fe3d40:cinder:volumes', 'backend_state': 'up', 'volume_backend_name': 'ceph', 'replication_enabled': False, 'properties': {'thin_provisioning': {'title': 'Thin Provisioning', 'description': 'Sets thin provisioning.', 'type': 'boolean'}, 'compression': {'title': 'Compression', 'description': 'Enables compression.', 'type': 'boolean'}, 'qos': {'title': 'QoS', 'description': 'Enables QoS.', 'type': 'boolean'}, 'replication_enabled': {'title': 'Replication', 'description': 'Enables replication.', 'type': 'boolean'}}}. {{(pid=156391) init_capabilities /opt/stack/cinder/cinder/volume/driver.py:812}}
Jul 02 21:18:46.888012 ncdv-master-1 cinder-volume[156391]: DEBUG oslo_db.sqlalchemy.engines [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=156391) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}}
Jul 02 21:18:46.891973 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writer thread running {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}}
Jul 02 21:18:47.007911 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.db.sqlalchemy.api [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Building query based on filter {{(pid=156391) _process_snaps_filters /opt/stack/cinder/cinder/db/sqlalchemy/api.py:3878}}
Jul 02 21:18:47.066967 ncdv-master-1 cinder-volume[156391]: INFO cinder.keymgr.migration [None req-9bfb0aec-5921-432b-9ab7-52b122d81ebc None None] Not migrating encryption keys because the ConfKeyManager is still in use.
Jul 02 21:18:47.073663 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:47.141415 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:47.224253 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:18:47.271735 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Driver initialization completed successfully.
Jul 02 21:18:47.272420 ncdv-master-1 cinder-volume[156391]: INFO cinder.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Initiating service 3 cleanup
Jul 02 21:18:47.366318 ncdv-master-1 cinder-volume[156391]: INFO cinder.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Cleaning Volume with id 56999857-0213-428a-a613-de4e9f99f593 and status creating
Jul 02 21:18:47.392961 ncdv-master-1 cinder-volume[156391]: INFO cinder.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Service 3 cleanup completed.
Jul 02 21:18:47.393652 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.service [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Creating RPC server for service cinder-volume {{(pid=156391) start /opt/stack/cinder/cinder/service.py:231}}
Jul 02 21:18:47.404715 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.service [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Pinning object versions for RPC server serializer to 1.39 {{(pid=156391) start /opt/stack/cinder/cinder/service.py:237}}
Jul 02 21:18:47.458012 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Initializing RPC dependent components of volume driver RBDDriver (1.2.0)
Jul 02 21:18:47.459160 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:47.507932 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:47.576330 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-8aefc4b1-8aa9-4b1b-a4f4-86d854897367 None None] Driver post RPC initialization completed successfully.
Jul 02 21:18:48.985102 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-8f4315b0-aae6-46f9-8be1-f3bf1afda405 req-8500a168-f01d-446b-bcd4-90268e70f345 tempest-AttachVolumeShelveTestJSON-1224937377 None] No connector for attachment b5f66443-559f-44b7-9b1b-47cbcaffc5be; skipping storage backend terminate_connection call. {{(pid=156391) _connection_terminate /opt/stack/cinder/cinder/volume/manager.py:4913}}
Jul 02 21:18:48.985768 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-8f4315b0-aae6-46f9-8be1-f3bf1afda405 req-8500a168-f01d-446b-bcd4-90268e70f345 tempest-AttachVolumeShelveTestJSON-1224937377 None] Deleting attachment b5f66443-559f-44b7-9b1b-47cbcaffc5be. {{(pid=156391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:18:55.348704 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-07b7b40b-8a38-43be-ad2b-41155338f8f2 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:55.804337 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-07b7b40b-8a38-43be-ad2b-41155338f8f2 tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:18:56.697660 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-3cf8edf9-c602-4202-ae86-55e4f318ff4c-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:18:56.733858 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-3cf8edf9-c602-4202-ae86-55e4f318ff4c-delete_snapshot" acquired by "delete_snapshot" :: waited 0.035s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:18:56.737768 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:18:57.959188 ncdv-master-1 cinder-volume[156391]: WARNING cinder.quota [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Deprecated: Default quota for resource: gigabytes_ceph is set by the default quota flag: quota_gigabytes_ceph, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:18:57.960214 ncdv-master-1 cinder-volume[156391]: WARNING cinder.quota [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Deprecated: Default quota for resource: snapshots_ceph is set by the default quota flag: quota_snapshots_ceph, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:18:57.996613 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['46ada8e6-31b1-4976-8160-3d1a6068c065', 'f44fc72a-34ee-494d-89f0-f36d25ead96d', '2a0512a2-ef5b-4175-8ccd-3530f7dbfafb', 'e6ee63cb-40b6-4622-b620-76fb68e163a4'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:18:58.180324 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:18:58.188848 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-18f65e7f-6d6b-4fe4-9caa-2b432daddda1 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-3cf8edf9-c602-4202-ae86-55e4f318ff4c-delete_snapshot" released by "delete_snapshot" :: held 1.456s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:18:59.379085 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-c45078ff-73d7-44ab-b611-5231ff84409b tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:19:00.102881 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-c45078ff-73d7-44ab-b611-5231ff84409b tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:19:17.598951 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=33,cinder:UPDATE=13,cinder:INSERT=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:19:17.602742 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=38,cinder:UPDATE=6,cinder:DELETE=3,cinder:INSERT=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:19:30.877001 ncdv-master-1 cinder-volume[156391]: DEBUG oslo_concurrency.processutils [req-3600f952-3e31-4017-a0ea-71677959bf91 req-e74d0f35-a0c7-4a0e-9d17-b6a5e1bea85a tempest-VolumesSnapshotTestJSON-1740353321 None] Running cmd (subprocess): ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf {{(pid=156391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:384}}
Jul 02 21:19:32.268304 ncdv-master-1 cinder-volume[156391]: DEBUG oslo_concurrency.processutils [req-3600f952-3e31-4017-a0ea-71677959bf91 req-e74d0f35-a0c7-4a0e-9d17-b6a5e1bea85a tempest-VolumesSnapshotTestJSON-1740353321 None] CMD "ceph mon dump --format=json --id cinder --cluster ceph --conf /etc/ceph/ceph.conf" returned: 0 in 1.394s {{(pid=156391) execute /usr/local/lib/python3.8/dist-packages/oslo_concurrency/processutils.py:422}}
Jul 02 21:19:32.269944 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-3600f952-3e31-4017-a0ea-71677959bf91 req-e74d0f35-a0c7-4a0e-9d17-b6a5e1bea85a tempest-VolumesSnapshotTestJSON-1740353321 None] connection data: {'driver_volume_type': 'rbd', 'data': {'name': 'volumes/volume-792de71d-1208-4d3b-90d3-efdd17ab60ab', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': 'ceph', 'secret_uuid': 'bf7f2f0c-b9c4-4629-91bb-9884bc3ae388', 'volume_id': '792de71d-1208-4d3b-90d3-efdd17ab60ab', 'discard': True}} {{(pid=156391) initialize_connection /opt/stack/cinder/cinder/volume/drivers/rbd.py:1649}}
Jul 02 21:19:32.346484 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-3600f952-3e31-4017-a0ea-71677959bf91 req-e74d0f35-a0c7-4a0e-9d17-b6a5e1bea85a tempest-VolumesSnapshotTestJSON-1740353321 None] Connection info returned from driver {'name': 'volumes/volume-792de71d-1208-4d3b-90d3-efdd17ab60ab', 'hosts': ['10.222.0.43'], 'ports': ['6789'], 'cluster_name': 'ceph', 'auth_enabled': True, 'auth_username': 'cinder', 'secret_type': '***', 'secret_uuid': '***', 'volume_id': '792de71d-1208-4d3b-90d3-efdd17ab60ab', 'discard': True, 'qos_specs': None, 'access_mode': 'rw', 'encrypted': False, 'cacheable': False, 'driver_volume_type': 'rbd', 'attachment_id': '77751d68-a102-4757-9388-4bbd684777c8'} {{(pid=156391) _connection_create /opt/stack/cinder/cinder/volume/manager.py:4827}}
Jul 02 21:19:32.427220 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-3600f952-3e31-4017-a0ea-71677959bf91 req-e74d0f35-a0c7-4a0e-9d17-b6a5e1bea85a tempest-VolumesSnapshotTestJSON-1740353321 None] attachment_update completed successfully.
Jul 02 21:19:35.446629 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-99f80454-6944-41d2-9686-7cce001a098c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:19:36.294126 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-99f80454-6944-41d2-9686-7cce001a098c tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:19:36.988516 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-5416e5cc-fffb-4e5a-9a77-42b45a2735e8 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:19:37.327666 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-5416e5cc-fffb-4e5a-9a77-42b45a2735e8 tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:19:38.026209 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-679c1187-c3a7-4c73-aea2-e12a2f789ccd tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-18454a0b-572c-41a6-b285-11f866a737db-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:19:38.054650 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-679c1187-c3a7-4c73-aea2-e12a2f789ccd tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-18454a0b-572c-41a6-b285-11f866a737db-delete_snapshot" acquired by "delete_snapshot" :: waited 0.028s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:19:38.057311 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-679c1187-c3a7-4c73-aea2-e12a2f789ccd tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:19:47.581426 ncdv-master-1 cinder-volume[156391]: DEBUG oslo_service.periodic_task [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=156391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:19:47.582792 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:19:47.649136 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:19:47.706304 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:19:47.728224 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=27,cinder:UPDATE=4,cinder:DELETE=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:19:47.728224 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=23,cinder:UPDATE=6 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:20:07.728156 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=4,cinder:UPDATE=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:20:08.350245 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-679c1187-c3a7-4c73-aea2-e12a2f789ccd tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['1520a093-cf14-474a-9574-5863fafe7f1f', '89811ca2-e650-4af3-96e2-685dc87a5841', '0d5efff0-f43c-4a88-a435-fd21cd36e39f', 'd2601256-65e1-480f-b095-e938399388f5'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:20:08.430417 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-679c1187-c3a7-4c73-aea2-e12a2f789ccd tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:20:08.435505 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-679c1187-c3a7-4c73-aea2-e12a2f789ccd tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-18454a0b-572c-41a6-b285-11f866a737db-delete_snapshot" released by "delete_snapshot" :: held 30.381s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:08.738805 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-47d9063b-6b73-40aa-8deb-eaadf8c9fa7c tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-2f099370-c1a2-4b6b-983c-3fcc53a96911-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:08.751663 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-47d9063b-6b73-40aa-8deb-eaadf8c9fa7c tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-2f099370-c1a2-4b6b-983c-3fcc53a96911-delete_snapshot" acquired by "delete_snapshot" :: waited 0.013s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:08.754559 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-47d9063b-6b73-40aa-8deb-eaadf8c9fa7c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:10.153754 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-e6629302-d4e3-4190-a2a2-dcd00b0c4fe1 req-8adc6850-64fa-4dc7-a666-82b3b8475a60 tempest-TaggedBootDevicesTest-1312932753 None] Terminate volume connection completed successfully.
Jul 02 21:20:10.154319 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-e6629302-d4e3-4190-a2a2-dcd00b0c4fe1 req-8adc6850-64fa-4dc7-a666-82b3b8475a60 tempest-TaggedBootDevicesTest-1312932753 None] Deleting attachment aec6e3a4-21cd-42c2-9071-9bad04197d87. {{(pid=156391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:20:10.382904 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-e6629302-d4e3-4190-a2a2-dcd00b0c4fe1 req-9897039b-fc08-4ab3-ab10-3a0fe9cf75f9 tempest-TaggedBootDevicesTest-1312932753 None] Terminate volume connection completed successfully.
Jul 02 21:20:10.383745 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-e6629302-d4e3-4190-a2a2-dcd00b0c4fe1 req-9897039b-fc08-4ab3-ab10-3a0fe9cf75f9 tempest-TaggedBootDevicesTest-1312932753 None] Deleting attachment 012237dd-ce78-464f-b331-baf72883d8c7. {{(pid=156391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:20:10.954905 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-e6629302-d4e3-4190-a2a2-dcd00b0c4fe1 req-cc4e0b59-6087-48fb-ab9e-ac60595dcf32 tempest-TaggedBootDevicesTest-1312932753 None] Terminate volume connection completed successfully.
Jul 02 21:20:10.955491 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-e6629302-d4e3-4190-a2a2-dcd00b0c4fe1 req-cc4e0b59-6087-48fb-ab9e-ac60595dcf32 tempest-TaggedBootDevicesTest-1312932753 None] Deleting attachment 39bcae2c-ba03-4f6a-8f5b-387b1183c4b1. {{(pid=156391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:20:15.199892 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Acquiring lock "cinder-eb6fa52b-f4db-4f5b-80e7-c80177b5e350-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:15.229375 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Lock "cinder-eb6fa52b-f4db-4f5b-80e7-c80177b5e350-delete_volume" acquired by "delete_volume" :: waited 0.026s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:15.321353 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:15.463840 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:20:15.468429 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350 is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:20:15.481338 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] deleting rbd volume volume-eb6fa52b-f4db-4f5b-80e7-c80177b5e350 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:20:15.853910 ncdv-master-1 cinder-volume[156391]: WARNING cinder.quota [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Deprecated: Default quota for resource: volumes_ceph is set by the default quota flag: quota_volumes_ceph, it is now deprecated. Please use the default quota class for default quota.
Jul 02 21:20:15.873061 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Created reservations ['88506968-5c7d-47a9-a8fd-0824adfde23e', '7d083b94-fe92-41a4-8ed0-3096dc104720', 'f7994159-f1c2-4d8e-82b0-cb4347888484', '3f05b372-f238-451c-b8ea-c3aedb98ac56'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:20:15.938081 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:16.004718 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:16.044548 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:20:16.051829 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Deleted volume successfully.
Jul 02 21:20:16.080252 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-83e3b17e-1ffd-478b-81b9-c30cf9a16325 tempest-AttachVolumeShelveTestJSON-1224937377 None] Lock "cinder-eb6fa52b-f4db-4f5b-80e7-c80177b5e350-delete_volume" released by "delete_volume" :: held 0.854s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:16.659643 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Acquiring lock "cinder-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:16.675694 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Lock "cinder-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:16.737155 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:16.787752 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:20:16.788463 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Volume volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342 is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:20:16.796316 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] deleting rbd volume volume-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:20:17.348524 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Created reservations ['ae41e15f-3985-45ed-8a8a-02fb3665521a', 'ec644634-8026-4db5-92cb-340f5d6c01c3', '129daa22-254e-48eb-a515-9f6a74111d1f', '53ffaf11-cbd1-4ae1-89df-a0c22cea51b6'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:20:17.447799 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:17.514622 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:17.554352 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:20:17.560994 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Deleted volume successfully.
Jul 02 21:20:17.589365 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-dd5ff770-9d75-4770-935d-b18f0cae083c tempest-AttachVolumeShelveTestJSON-1224937377 None] Lock "cinder-aa1b5e5c-92f9-4ce4-b42d-2aeafada2342-delete_volume" released by "delete_volume" :: held 0.914s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:24.161339 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Acquiring lock "cinder-6593f24e-5231-490f-8bfd-665de85ebc5b-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:24.177677 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-6593f24e-5231-490f-8bfd-665de85ebc5b-delete_volume" acquired by "delete_volume" :: waited 0.016s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:24.197374 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=47,cinder:UPDATE=24,cinder:INSERT=5,cinder:DELETE=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:20:24.239216 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:24.356425 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:20:24.357304 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Volume volume-6593f24e-5231-490f-8bfd-665de85ebc5b is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:20:24.368320 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] deleting rbd volume volume-6593f24e-5231-490f-8bfd-665de85ebc5b {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:20:24.753848 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Created reservations ['55b7c034-20f7-43f3-a651-f15f2cf65a49', '51d32671-3694-40c3-a739-f348de553997', '04e07651-ca11-48ce-8f29-455ae1b3defe', '9452e088-7586-4a34-a730-8e0b1b414415'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:20:24.818837 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:24.908731 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:24.969312 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:20:24.976902 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Deleted volume successfully.
Jul 02 21:20:25.000182 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-d3a0f993-19e0-4173-a081-649b3eca2e43 tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-6593f24e-5231-490f-8bfd-665de85ebc5b-delete_volume" released by "delete_volume" :: held 0.822s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:25.441743 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Acquiring lock "cinder-a22e0b4d-265f-434c-bfd9-e859326dc395-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:25.494835 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-a22e0b4d-265f-434c-bfd9-e859326dc395-delete_volume" acquired by "delete_volume" :: waited 0.054s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:25.545138 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=63,cinder:INSERT=7,cinder:UPDATE=30 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:20:25.567886 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:25.633183 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:20:25.633682 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Volume volume-a22e0b4d-265f-434c-bfd9-e859326dc395 is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:20:25.642796 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] deleting rbd volume volume-a22e0b4d-265f-434c-bfd9-e859326dc395 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:20:26.143346 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Created reservations ['35a55fa0-93a5-495c-a6df-b49bf64d4558', '5704b17f-6099-437c-8519-063619e00210', '681d69b3-6d87-4f7f-b5fc-4619b443db16', 'be2b5cad-67a2-4de8-92aa-45cc02dd8916'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:20:26.209717 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:26.278540 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:26.349683 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:20:26.358179 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Deleted volume successfully.
Jul 02 21:20:26.387030 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-999d1ab4-6dab-4a0b-8d5f-5c899f3e065a tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-a22e0b4d-265f-434c-bfd9-e859326dc395-delete_volume" released by "delete_volume" :: held 0.892s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:26.682429 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] Acquiring lock "cinder-4f464256-167f-499c-a620-977a48ba0220-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:26.701009 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-4f464256-167f-499c-a620-977a48ba0220-delete_volume" acquired by "delete_volume" :: waited 0.019s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:26.764888 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:26.862058 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:20:26.874329 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] deleting rbd volume volume-4f464256-167f-499c-a620-977a48ba0220 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:20:27.389595 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] volume is a clone so cleaning references {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1346}}
Jul 02 21:20:27.427112 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] deleting parent snapshot volume-4f464256-167f-499c-a620-977a48ba0220.clone_snap {{(pid=156391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}}
Jul 02 21:20:27.552082 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] Lock "cinder-4f464256-167f-499c-a620-977a48ba0220-delete_volume" released by "delete_volume" :: held 0.851s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server [None req-5af4f0a2-3a90-4cd7-914e-583dbf529941 tempest-TaggedBootDevicesTest-1312932753 None] Exception during message handling: rbd.ReadOnlyImage: [errno 30] RBD read-only image (error removing snapshot b'volume-4f464256-167f-499c-a620-977a48ba0220.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 203, in wrapper
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     skip_clean = func(self, context, volume, *args, **kwargs)
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return caller(func, *(extras + args), **kw)
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/coordination.py", line 197, in _synchronized
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return f(*a, **k)
Jul 02 21:20:27.636013 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return caller(func, *(extras + args), **kw)
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/objects/cleanable.py", line 208, in wrapper
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 1002, in delete_volume
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self._clear_db(volume, new_status)
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     raise self.value
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 986, in delete_volume
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self.driver.delete_volume(volume)
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 1348, in delete_volume
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self._delete_clone_parent_refs(client, parent, parent_snap)
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 1241, in _delete_clone_parent_refs
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     parent_rbd.remove_snap(parent_snap)
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3551, in rbd.Image.remove_snap
Jul 02 21:20:27.637171 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server rbd.ReadOnlyImage: [errno 30] RBD read-only image (error removing snapshot b'volume-4f464256-167f-499c-a620-977a48ba0220.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:20:27.638381 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:20:39.055445 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-47d9063b-6b73-40aa-8deb-eaadf8c9fa7c tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['6964b622-a4e8-4f01-bcbc-b24259dea28d', '6b8fa285-e594-430f-905d-242195d3bf0d', '73595002-98b6-4dff-97cc-00c9dfe146c6', 'bfc50e47-6008-4a99-ae8c-7c98ea8c9cf2'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:20:39.136922 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-47d9063b-6b73-40aa-8deb-eaadf8c9fa7c tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:20:39.143001 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-47d9063b-6b73-40aa-8deb-eaadf8c9fa7c tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-2f099370-c1a2-4b6b-983c-3fcc53a96911-delete_snapshot" released by "delete_snapshot" :: held 30.391s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:20:39.352214 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-88547329-f7f7-48fa-b80d-fd96f1d3f161 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-8e38007a-8906-4d00-bad1-955109250ba8-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:20:39.367886 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-88547329-f7f7-48fa-b80d-fd96f1d3f161 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-8e38007a-8906-4d00-bad1-955109250ba8-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:20:39.379961 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-88547329-f7f7-48fa-b80d-fd96f1d3f161 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:47.582996 ncdv-master-1 cinder-volume[156391]: DEBUG oslo_service.periodic_task [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=156391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:20:47.583661 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:47.654081 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:20:47.728963 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:20:57.742102 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=28,cinder:INSERT=1,cinder:UPDATE=22,cinder:DELETE=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:20:57.743294 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=35,cinder:INSERT=11,cinder:UPDATE=17,cinder:DELETE=1 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:07.789030 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=1,cinder:UPDATE=1 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:08.838717 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:09.039038 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Flow 'volume_create_manager' (06b14244-70fb-4ecf-8f73-c18e4c047f95) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:09.054838 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (859132d4-579a-49df-99b7-a12fa683b370) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:09.078744 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (859132d4-579a-49df-99b7-a12fa683b370) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:08Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesGetTestJSON-Volume-1506108419',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=79a7d33a-dc8e-441e-b734-71a414636ddb,launched_at=None,metadata={Type='work'},migration_status=None,multiattach=False,previous_status=None,project_id='0e3d8ce95c454de9bea4c3a1beff5678',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:21:09Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:21:09Z,use_quota=True,user_id='b89bdaabad8b49c789e5382df34c1cb2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:09.087261 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3818dc3d-84dc-438a-88c0-59eef664d25e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:09.094478 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (3818dc3d-84dc-438a-88c0-59eef664d25e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:09.101039 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08b4f6b3-d0fa-403b-9a4f-6bc1ec53f2c1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:09.110960 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (08b4f6b3-d0fa-403b-9a4f-6bc1ec53f2c1) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '79a7d33a-dc8e-441e-b734-71a414636ddb', 'volume_name': 'volume-79a7d33a-dc8e-441e-b734-71a414636ddb', 'volume_size': 1}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:09.119869 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3635be00-c9cb-435f-b364-0abb331b87dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:09.127048 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3635be00-c9cb-435f-b364-0abb331b87dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:09.135154 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (637af02a-e83e-490a-9fae-e60803f61ae0) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:09.136750 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Volume 79a7d33a-dc8e-441e-b734-71a414636ddb: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-79a7d33a-dc8e-441e-b734-71a414636ddb', 'volume_size': 1}
Jul 02 21:21:09.137230 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] creating volume 'volume-79a7d33a-dc8e-441e-b734-71a414636ddb' {{(pid=156391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:21:09.143991 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:09.269125 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:21:09.269779 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:21:09.273703 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (637af02a-e83e-490a-9fae-e60803f61ae0) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-79a7d33a-dc8e-441e-b734-71a414636ddb', 'volume_size': 1}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:09.285544 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c87b29c-3c6b-4d98-a759-8af7afc1d4dd) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:09.320622 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Volume volume-79a7d33a-dc8e-441e-b734-71a414636ddb (79a7d33a-dc8e-441e-b734-71a414636ddb): created successfully
Jul 02 21:21:09.326354 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5c87b29c-3c6b-4d98-a759-8af7afc1d4dd) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:09.332389 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Flow 'volume_create_manager' (06b14244-70fb-4ecf-8f73-c18e4c047f95) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:09.349622 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-20e02322-839b-4e99-be1d-d5f106a143c9 req-0481a1c9-b7bd-4707-a2c0-41b715476f8b tempest-VolumesGetTestJSON-642642729 None] Created volume successfully.
Jul 02 21:21:09.610880 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-88547329-f7f7-48fa-b80d-fd96f1d3f161 tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['a2c0f6bc-1cbb-4f6c-ac63-fbdc50c52070', '13cfd27c-6b06-49bc-a568-69d0df0c5c76', 'ce666a45-a848-4a71-9cba-9e3d301b4a7d', '7f2ab35a-805c-44f1-9132-7589ed2b97e5'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:09.697577 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-88547329-f7f7-48fa-b80d-fd96f1d3f161 tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:21:09.703688 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-88547329-f7f7-48fa-b80d-fd96f1d3f161 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-8e38007a-8906-4d00-bad1-955109250ba8-delete_snapshot" released by "delete_snapshot" :: held 30.336s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:10.666678 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Acquiring lock "cinder-79a7d33a-dc8e-441e-b734-71a414636ddb-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:10.684974 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Lock "cinder-79a7d33a-dc8e-441e-b734-71a414636ddb-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:10.749752 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:10.834125 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:10.835512 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Volume volume-79a7d33a-dc8e-441e-b734-71a414636ddb is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:21:10.841343 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] deleting rbd volume volume-79a7d33a-dc8e-441e-b734-71a414636ddb {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:11.269541 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Created reservations ['685d40e6-6ba5-46e4-82fd-91e78f5f9655', '01bab37c-6938-4491-9892-7d75c94391dc', 'f666385a-3c48-471f-8e59-ba0a20e419e6', 'bb809680-d3f4-4787-8b53-b5dcfd3065e5'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:11.405544 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:11.470559 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:11.574241 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:11.581813 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Deleted volume successfully.
Jul 02 21:21:11.631325 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [req-b1b5e832-d8be-4993-b4f1-99d7f8eb4424 req-7f84fd66-5c4e-4a2f-8bd8-deca13a06cea tempest-VolumesGetTestJSON-642642729 None] Lock "cinder-79a7d33a-dc8e-441e-b734-71a414636ddb-delete_volume" released by "delete_volume" :: held 0.946s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:12.306055 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [req-2f8e8982-15ce-4bda-9806-bee3118bf376 req-2f03d5b3-5c53-4050-a964-316b34bb4e14 tempest-VolumesSnapshotTestJSON-1740353321 None] Terminate volume connection completed successfully.
Jul 02 21:21:12.306674 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [req-2f8e8982-15ce-4bda-9806-bee3118bf376 req-2f03d5b3-5c53-4050-a964-316b34bb4e14 tempest-VolumesSnapshotTestJSON-1740353321 None] Deleting attachment 77751d68-a102-4757-9388-4bbd684777c8. {{(pid=156391) attachment_delete /opt/stack/cinder/cinder/volume/manager.py:4958}}
Jul 02 21:21:20.844661 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:20.869032 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (339bc1de-0c9e-4867-bef4-e5b19ab92925) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:20.876779 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0dfd11e4-0ba7-482c-9292-aaecc307bd86) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:20.900685 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (0dfd11e4-0ba7-482c-9292-aaecc307bd86) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:20Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1086910779',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c7534889ba5b48338c27c0556be3c6f0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:21:21Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:21:21Z,use_quota=True,user_id='2016769bf3a34d22992b45f9974b99a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:20.907007 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a35a7865-fd36-42a9-9a39-8785fbedf9f3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:20.914225 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (a35a7865-fd36-42a9-9a39-8785fbedf9f3) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:20.924115 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e6cd3be2-5577-4277-a905-50e249459057) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:20.930010 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (e6cd3be2-5577-4277-a905-50e249459057) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe', 'volume_name': 'volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe', 'volume_size': 1}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:20.938571 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1f37754f-070e-4c72-9644-67cb5cd61d73) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:20.944198 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (1f37754f-070e-4c72-9644-67cb5cd61d73) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:20.955449 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (227cdab1-22c3-4930-8109-b78c80fdceb2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:20.956853 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Volume 9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe', 'volume_size': 1}
Jul 02 21:21:20.957431 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] creating volume 'volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe' {{(pid=156391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:21:20.958541 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:21.048768 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:21:21.049553 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:21:21.057630 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (227cdab1-22c3-4930-8109-b78c80fdceb2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe', 'volume_size': 1}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:21.066116 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (272302a0-fb37-4f98-892f-e01eabe7bc6c) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:21.098844 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe (9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe): created successfully
Jul 02 21:21:21.103822 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (272302a0-fb37-4f98-892f-e01eabe7bc6c) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:21.115716 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (339bc1de-0c9e-4867-bef4-e5b19ab92925) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:21.143443 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-023d9342-49f6-46c5-ba41-948f551b494c tempest-VolumesSnapshotTestJSON-1740353321 None] Created volume successfully.
Jul 02 21:21:22.272808 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-49008dd7-0790-4c3e-8868-4e9725260b1f tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:23.162694 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-49008dd7-0790-4c3e-8868-4e9725260b1f tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:21:24.768747 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:24.806276 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (07d2c7cd-9fc8-46e7-bf46-60d80ac5bd55) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:24.812975 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1fa55bd4-99e7-437b-b9ff-b443e2ea1046) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:24.831050 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1fa55bd4-99e7-437b-b9ff-b443e2ea1046) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:24Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1120873981',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=3845c6da-6ed8-44ad-838f-d21322c17c31,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c7534889ba5b48338c27c0556be3c6f0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:21:25Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=8fdd150b-474d-4df9-a9c6-727e383d3268,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:21:25Z,use_quota=True,user_id='2016769bf3a34d22992b45f9974b99a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:24.839573 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4d5cd109-9e8d-4b02-9e81-56d06fd39207) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:24.845233 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (4d5cd109-9e8d-4b02-9e81-56d06fd39207) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:24.850710 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6f7b8ccb-da04-4144-982c-148f8f3e6e90) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:24.856930 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (6f7b8ccb-da04-4144-982c-148f8f3e6e90) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '3845c6da-6ed8-44ad-838f-d21322c17c31', 'volume_name': 'volume-3845c6da-6ed8-44ad-838f-d21322c17c31', 'volume_size': 2, 'snapshot_id': '8fdd150b-474d-4df9-a9c6-727e383d3268'}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:24.862218 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d50c4dc2-8682-4581-89e2-4d2b383cd108) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:24.866187 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d50c4dc2-8682-4581-89e2-4d2b383cd108) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:24.870698 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (409baf72-4201-495f-83c3-a4b1fb56e0de) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:24.871772 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume 3845c6da-6ed8-44ad-838f-d21322c17c31: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-3845c6da-6ed8-44ad-838f-d21322c17c31', 'volume_size': 2, 'snapshot_id': '8fdd150b-474d-4df9-a9c6-727e383d3268'}
Jul 02 21:21:24.916917 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] cloning volumes/volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe@snapshot-8fdd150b-474d-4df9-a9c6-727e383d3268 to volume-3845c6da-6ed8-44ad-838f-d21322c17c31 {{(pid=156391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}}
Jul 02 21:21:24.917642 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:24.962635 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:25.112028 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:21:25.112823 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:21:25.124795 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:25.357149 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:25.457637 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.drivers.rbd [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Not using v2 clone API, please upgrade to mimic+ and set the OSD minimum client compat version to mimic for better performance, fewer deletion issues
Jul 02 21:21:25.508383 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (409baf72-4201-495f-83c3-a4b1fb56e0de) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-3845c6da-6ed8-44ad-838f-d21322c17c31', 'volume_size': 2, 'snapshot_id': '8fdd150b-474d-4df9-a9c6-727e383d3268'}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:25.518755 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d69164e6-a78b-4be5-9d5a-ce239f7752c6) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:25.540911 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-3845c6da-6ed8-44ad-838f-d21322c17c31 (3845c6da-6ed8-44ad-838f-d21322c17c31): created successfully
Jul 02 21:21:25.547157 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (d69164e6-a78b-4be5-9d5a-ce239f7752c6) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:25.558254 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (07d2c7cd-9fc8-46e7-bf46-60d80ac5bd55) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:25.579348 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-b280fa2a-9402-48be-bf8a-c7abf99d3628 tempest-VolumesSnapshotTestJSON-1740353321 None] Created volume successfully.
Jul 02 21:21:25.951833 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-3845c6da-6ed8-44ad-838f-d21322c17c31-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:25.969164 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-3845c6da-6ed8-44ad-838f-d21322c17c31-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:26.078180 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:26.191179 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:26.207713 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-3845c6da-6ed8-44ad-838f-d21322c17c31 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:26.624455 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['743ce1bb-d3f8-475a-9af1-9a5270cdb996', 'fb5eb742-340c-4ae2-909a-9e213a11f91f', 'c3468e7b-7e8f-4e9c-8379-42074c07e6bf', '1a40956f-aa82-4fd5-9af3-cd64e3876aa8'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:26.725652 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:26.804246 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:26.883846 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:26.895538 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] Deleted volume successfully.
Jul 02 21:21:26.918988 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-ab762910-3162-4f33-8469-fa0aee806c3c tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-3845c6da-6ed8-44ad-838f-d21322c17c31-delete_volume" released by "delete_volume" :: held 0.950s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:27.742394 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=62,cinder:UPDATE=32,cinder:DELETE=2,cinder:INSERT=4 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:27.833464 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:27.875981 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (4cc52e11-57f9-4fc0-a2b8-9c6755b80ad1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:27.885764 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a205efe1-9bdf-4358-9036-d15fc6a6fd3e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:27.975748 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (a205efe1-9bdf-4358-9036-d15fc6a6fd3e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:27Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-1996661236',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=a9e77abb-e3df-4f04-96d8-d3aacf52bf9f,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c7534889ba5b48338c27c0556be3c6f0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:21:28Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:21:28Z,use_quota=True,user_id='2016769bf3a34d22992b45f9974b99a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:27.981278 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (efd6bb2d-a277-49aa-954a-6e56bf309966) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:27.998680 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (efd6bb2d-a277-49aa-954a-6e56bf309966) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:28.012417 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05d58ed6-b561-41f6-a220-cc9243163c2d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:28.028170 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (05d58ed6-b561-41f6-a220-cc9243163c2d) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': 'a9e77abb-e3df-4f04-96d8-d3aacf52bf9f', 'volume_name': 'volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f', 'volume_size': 1}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:28.042606 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (265f22ce-fbac-4a4f-a466-6266837d7171) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:28.048822 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (265f22ce-fbac-4a4f-a466-6266837d7171) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:28.060471 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c415a6f7-5496-4aff-98fa-894f0771eb72) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:28.066177 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume a9e77abb-e3df-4f04-96d8-d3aacf52bf9f: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f', 'volume_size': 1}
Jul 02 21:21:28.066177 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] creating volume 'volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f' {{(pid=156391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:21:28.066177 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:28.205757 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:21:28.206143 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:21:28.210744 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (c415a6f7-5496-4aff-98fa-894f0771eb72) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f', 'volume_size': 1}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:28.215121 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (72386628-2ff9-4460-8993-c2e834f71d44) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:28.244381 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f (a9e77abb-e3df-4f04-96d8-d3aacf52bf9f): created successfully
Jul 02 21:21:28.248890 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (72386628-2ff9-4460-8993-c2e834f71d44) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:28.254573 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (4cc52e11-57f9-4fc0-a2b8-9c6755b80ad1) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:28.272621 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-37eaf400-cd62-4d19-a0f5-9a746e6a6721 tempest-VolumesSnapshotTestJSON-1740353321 None] Created volume successfully.
Jul 02 21:21:29.035761 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=61,cinder:UPDATE=26,cinder:INSERT=8,cinder:DELETE=5 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:29.095298 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-1c75a9bd-932b-4849-b2e3-0cdd0d28151a tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:30.351888 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-1c75a9bd-932b-4849-b2e3-0cdd0d28151a tempest-VolumesSnapshotTestJSON-1740353321 None] Create snapshot completed successfully
Jul 02 21:21:31.972622 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:32.013026 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (c5d8e828-2a2a-452b-b320-b0cc4e4ae23b) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:32.021560 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b56b1d63-ad12-431e-8512-4f1e4c562c4a) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:32.041402 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (b56b1d63-ad12-431e-8512-4f1e4c562c4a) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:32Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotTestJSON-Volume-266274421',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=12cc088d-1149-4c3d-bde9-7ecef199528a,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='c7534889ba5b48338c27c0556be3c6f0',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:21:32Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=98059eeb-f1c5-4157-87fc-61f027a45bad,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:21:32Z,use_quota=True,user_id='2016769bf3a34d22992b45f9974b99a2',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:32.057773 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1a59dafd-b4f7-4083-8f22-739232a53aca) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:32.065101 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (1a59dafd-b4f7-4083-8f22-739232a53aca) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:32.071384 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d8ed4d6c-5ade-4a5e-86b7-a19645058713) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:32.076261 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (d8ed4d6c-5ade-4a5e-86b7-a19645058713) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'snap', 'volume_id': '12cc088d-1149-4c3d-bde9-7ecef199528a', 'volume_name': 'volume-12cc088d-1149-4c3d-bde9-7ecef199528a', 'volume_size': 1, 'snapshot_id': '98059eeb-f1c5-4157-87fc-61f027a45bad'}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:32.083194 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0bc015d9-f8a3-4f30-8ec1-7943de2b032d) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:32.085408 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (0bc015d9-f8a3-4f30-8ec1-7943de2b032d) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:32.100689 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d8df201f-8b9d-452a-b97e-670ae0c574cc) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:32.101842 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume 12cc088d-1149-4c3d-bde9-7ecef199528a: being created as snap with specification: {'status': 'creating', 'volume_name': 'volume-12cc088d-1149-4c3d-bde9-7ecef199528a', 'volume_size': 1, 'snapshot_id': '98059eeb-f1c5-4157-87fc-61f027a45bad'}
Jul 02 21:21:32.150370 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] cloning volumes/volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f@snapshot-98059eeb-f1c5-4157-87fc-61f027a45bad to volume-12cc088d-1149-4c3d-bde9-7ecef199528a {{(pid=156391) _clone /opt/stack/cinder/cinder/volume/drivers/rbd.py:1094}}
Jul 02 21:21:32.151391 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:32.264989 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:32.493547 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:21:32.494117 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:21:32.538811 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (d8df201f-8b9d-452a-b97e-670ae0c574cc) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-12cc088d-1149-4c3d-bde9-7ecef199528a', 'volume_size': 1, 'snapshot_id': '98059eeb-f1c5-4157-87fc-61f027a45bad'}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:32.544832 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5cc5c868-79fa-4657-912a-363f9dd2ea65) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:32.561935 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-12cc088d-1149-4c3d-bde9-7ecef199528a (12cc088d-1149-4c3d-bde9-7ecef199528a): created successfully
Jul 02 21:21:32.565759 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (5cc5c868-79fa-4657-912a-363f9dd2ea65) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:32.578416 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Flow 'volume_create_manager' (c5d8e828-2a2a-452b-b320-b0cc4e4ae23b) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:32.615291 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-1ac18058-ef24-4128-8491-0728a5831807 tempest-VolumesSnapshotTestJSON-1740353321 None] Created volume successfully.
Jul 02 21:21:33.297620 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-12cc088d-1149-4c3d-bde9-7ecef199528a-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:33.315411 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-12cc088d-1149-4c3d-bde9-7ecef199528a-delete_volume" acquired by "delete_volume" :: waited 0.018s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:33.451784 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:33.632443 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:33.646527 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-12cc088d-1149-4c3d-bde9-7ecef199528a {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:34.145914 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['9063bd29-b871-4b7f-b8e2-dd958c350451', 'f033c7f6-bea7-4c52-84a9-cd23f5671d91', 'f50c5641-f30c-42c0-a7dd-3f7022485196', '84d943c9-918a-47a9-89ad-614ba4319a24'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:34.261048 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:34.345174 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:34.406221 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:34.411085 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] Deleted volume successfully.
Jul 02 21:21:34.451203 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-41a65b7a-3199-4f65-b4e6-bede3036d4ff tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-12cc088d-1149-4c3d-bde9-7ecef199528a-delete_volume" released by "delete_volume" :: held 1.136s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:34.654788 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-89c824c2-29e6-43cc-af75-60ec3685b758 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-98059eeb-f1c5-4157-87fc-61f027a45bad-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:34.689124 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-89c824c2-29e6-43cc-af75-60ec3685b758 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-98059eeb-f1c5-4157-87fc-61f027a45bad-delete_snapshot" acquired by "delete_snapshot" :: waited 0.033s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:34.693781 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-89c824c2-29e6-43cc-af75-60ec3685b758 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:35.507478 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-89c824c2-29e6-43cc-af75-60ec3685b758 tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['83a67fc4-ca63-41f9-a9ca-b089ae1e6bce', 'fe4d004e-b1e0-4bbf-8354-e1dcdc159612', '70635e62-4aef-4fb2-8741-7858b36be4c6', 'de40f8a8-fb60-4036-89f3-cd6224107180'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:35.584760 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-89c824c2-29e6-43cc-af75-60ec3685b758 tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:21:35.589419 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-89c824c2-29e6-43cc-af75-60ec3685b758 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-98059eeb-f1c5-4157-87fc-61f027a45bad-delete_snapshot" released by "delete_snapshot" :: held 0.902s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:35.860115 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:35.870984 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f-delete_volume" acquired by "delete_volume" :: waited 0.013s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:35.965664 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:36.045171 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:36.047073 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:21:36.059029 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:36.474774 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['6ff8e58d-3b5a-4881-bbb1-c870d45beaa2', 'bcae6650-2bc5-4a29-951b-88ed35a12d4a', 'ef0e869f-6416-4a75-985f-e155c07ab326', '17637744-c529-4fdc-890b-e4bf93ea156e'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:36.544731 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:36.619580 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:36.672247 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:36.684521 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Deleted volume successfully.
Jul 02 21:21:36.709239 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-3bc9a6ca-ed60-4a62-be76-ce6ddd3a1cb1 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-a9e77abb-e3df-4f04-96d8-d3aacf52bf9f-delete_volume" released by "delete_volume" :: held 0.837s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:37.219137 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-e0a49584-0f56-482f-bc65-7b4ebe3dfa3f tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-8fdd150b-474d-4df9-a9c6-727e383d3268-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:37.236319 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-e0a49584-0f56-482f-bc65-7b4ebe3dfa3f tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-8fdd150b-474d-4df9-a9c6-727e383d3268-delete_snapshot" acquired by "delete_snapshot" :: waited 0.016s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:37.238045 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-e0a49584-0f56-482f-bc65-7b4ebe3dfa3f tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:38.566942 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-e0a49584-0f56-482f-bc65-7b4ebe3dfa3f tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['76917df1-7066-4796-b510-414355484343', '7fd66342-8b2e-43b3-a739-320d85f2f5af', 'a5eafcd1-7b3b-4f26-b21f-611c3cb6c2b2', '608394c9-5535-4bfa-b840-227ddd29a1a3'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:38.632716 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=52,cinder:DELETE=3,cinder:UPDATE=32,cinder:INSERT=13 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:38.645949 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-e0a49584-0f56-482f-bc65-7b4ebe3dfa3f tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:21:38.662269 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-e0a49584-0f56-482f-bc65-7b4ebe3dfa3f tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-8fdd150b-474d-4df9-a9c6-727e383d3268-delete_snapshot" released by "delete_snapshot" :: held 1.427s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:39.481890 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:39.498737 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe-delete_volume" acquired by "delete_volume" :: waited 0.017s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:39.571819 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=65,cinder:UPDATE=31,cinder:DELETE=1,cinder:INSERT=3 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:39.574700 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:39.678791 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:39.679347 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:21:39.690363 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:40.229041 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['dcba037c-c678-4455-bb51-c4f6b8550007', '20c3f3e9-2265-4020-bbd5-fd4954c3b3b6', 'c5b4fcc6-ebcd-4b7e-9fe6-25b0c64da55c', '23524303-3172-4fd8-80ea-3432b43f22d1'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:40.396821 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:40.465158 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:40.549496 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:40.554011 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Deleted volume successfully.
Jul 02 21:21:40.574907 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-7e841f1e-2146-4794-a638-2e0c734c0370 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-9f6d3fd0-fc4a-4698-9143-7b8d8d3583fe-delete_volume" released by "delete_volume" :: held 1.076s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:40.866617 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-56999857-0213-428a-a613-de4e9f99f593-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:40.907157 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-56999857-0213-428a-a613-de4e9f99f593-delete_volume" acquired by "delete_volume" :: waited 0.038s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:40.991241 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:41.118446 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:41.134953 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-56999857-0213-428a-a613-de4e9f99f593 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:41.564451 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] volume is a clone so cleaning references {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1346}}
Jul 02 21:21:41.610471 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] deleting parent snapshot volume-56999857-0213-428a-a613-de4e9f99f593.clone_snap {{(pid=156391) _delete_clone_parent_refs /opt/stack/cinder/cinder/volume/drivers/rbd.py:1239}}
Jul 02 21:21:41.722881 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-56999857-0213-428a-a613-de4e9f99f593-delete_volume" released by "delete_volume" :: held 0.818s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server [None req-3dc3cd30-c046-4511-a771-b2d81e0dba2c tempest-VolumesSnapshotTestJSON-1740353321 None] Exception during message handling: rbd.ReadOnlyImage: [errno 30] RBD read-only image (error removing snapshot b'volume-56999857-0213-428a-a613-de4e9f99f593.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 203, in wrapper
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     skip_clean = func(self, context, volume, *args, **kwargs)
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return caller(func, *(extras + args), **kw)
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/coordination.py", line 197, in _synchronized
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return f(*a, **k)
Jul 02 21:21:41.724211 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return caller(func, *(extras + args), **kw)
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/objects/cleanable.py", line 208, in wrapper
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 1002, in delete_volume
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self._clear_db(volume, new_status)
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self.force_reraise()
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     raise self.value
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 986, in delete_volume
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self.driver.delete_volume(volume)
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 1348, in delete_volume
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     self._delete_clone_parent_refs(client, parent, parent_snap)
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 1241, in _delete_clone_parent_refs
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     parent_rbd.remove_snap(parent_snap)
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3551, in rbd.Image.remove_snap
Jul 02 21:21:41.725288 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server rbd.ReadOnlyImage: [errno 30] RBD read-only image (error removing snapshot b'volume-56999857-0213-428a-a613-de4e9f99f593.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:21:41.726334 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:21:42.130797 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-54ffdf69-628b-45a3-a01c-84a0388bb91a tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-7f1b0b27-ec0f-4fdc-ae9c-b9eb4c143e6d-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:42.149697 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-54ffdf69-628b-45a3-a01c-84a0388bb91a tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-7f1b0b27-ec0f-4fdc-ae9c-b9eb4c143e6d-delete_snapshot" acquired by "delete_snapshot" :: waited 0.019s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:42.161443 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-54ffdf69-628b-45a3-a01c-84a0388bb91a tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:42.894274 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-54ffdf69-628b-45a3-a01c-84a0388bb91a tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['002a1d47-ec5a-4945-9c21-585361654a08', 'a33b815e-33a4-4f81-8a83-5a2df46ed130', '0e190c52-84c4-468f-8580-ade34d322314', 'f4e28f44-2690-42ed-883d-552d24fa5c1b'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:43.000754 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-54ffdf69-628b-45a3-a01c-84a0388bb91a tempest-VolumesSnapshotTestJSON-1740353321 None] Delete snapshot completed successfully.
Jul 02 21:21:43.009731 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-54ffdf69-628b-45a3-a01c-84a0388bb91a tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-7f1b0b27-ec0f-4fdc-ae9c-b9eb4c143e6d-delete_snapshot" released by "delete_snapshot" :: held 0.858s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:43.325354 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Acquiring lock "cinder-792de71d-1208-4d3b-90d3-efdd17ab60ab-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:43.337020 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-792de71d-1208-4d3b-90d3-efdd17ab60ab-delete_volume" acquired by "delete_volume" :: waited 0.012s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:43.402435 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:43.490368 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:43.490803 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Volume volume-792de71d-1208-4d3b-90d3-efdd17ab60ab is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:21:43.498610 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] deleting rbd volume volume-792de71d-1208-4d3b-90d3-efdd17ab60ab {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:43.896371 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Created reservations ['ce8cb254-4bed-4685-9497-e2a46dd8c4d2', 'a71420ce-c5d8-4fca-9b37-5eae12bf0182', '8dadfc3c-eff5-422f-9c5a-fc194c43f4fb', '1e1c07cd-c37c-467f-9a17-2150d848a9b5'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:44.015127 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:44.105813 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:44.177346 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:44.184685 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Deleted volume successfully.
Jul 02 21:21:44.206896 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-a77d8b5b-5a7c-417a-aaa2-20e31c469c66 tempest-VolumesSnapshotTestJSON-1740353321 None] Lock "cinder-792de71d-1208-4d3b-90d3-efdd17ab60ab-delete_volume" released by "delete_volume" :: held 0.869s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:47.579330 ncdv-master-1 cinder-volume[156391]: DEBUG oslo_service.periodic_task [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] Running periodic task VolumeManager.publish_service_capabilities {{(pid=156391) run_periodic_tasks /usr/local/lib/python3.8/dist-packages/oslo_service/periodic_task.py:210}}
Jul 02 21:21:47.581630 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:47.683035 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:47.804939 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-af14a54e-3a37-46cd-9f10-fd7a9c4af22a None None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:52.399337 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:52.459633 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Flow 'volume_create_manager' (bf7c7c88-c6eb-4826-afac-549081bc9995) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:52.486451 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4ca37fc9-0bad-423a-af6b-c4be1f255eed) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:52.516128 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (4ca37fc9-0bad-423a-af6b-c4be1f255eed) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:52Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesSnapshotNegativeTestJSON-Volume-1906133826',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=1cf87253-ec46-4eb5-ae6a-431442424c10,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='e78773926d24460697194be02c372a33',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:21:52Z,service_uuid=None,shared_targets=True,size=2,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:21:52Z,use_quota=True,user_id='d64dd6bef28d4da1a51dfed953b24e05',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:52.522037 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (66d934fe-62c9-4ebb-ab09-42098b2f7eef) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:52.526579 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (66d934fe-62c9-4ebb-ab09-42098b2f7eef) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:52.550880 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ec6f51c7-6e09-4d6d-8b95-8cb2e880c480) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:52.561085 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (ec6f51c7-6e09-4d6d-8b95-8cb2e880c480) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'raw', 'volume_id': '1cf87253-ec46-4eb5-ae6a-431442424c10', 'volume_name': 'volume-1cf87253-ec46-4eb5-ae6a-431442424c10', 'volume_size': 2}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:52.571951 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d03951e6-6036-43f8-b10c-453195a7d890) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:52.585148 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (d03951e6-6036-43f8-b10c-453195a7d890) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:52.597707 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dc9a72df-7a24-4c44-a224-0a0dfc36c9b3) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:52.619534 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Volume 1cf87253-ec46-4eb5-ae6a-431442424c10: being created as raw with specification: {'status': 'creating', 'volume_name': 'volume-1cf87253-ec46-4eb5-ae6a-431442424c10', 'volume_size': 2}
Jul 02 21:21:52.622342 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] creating volume 'volume-1cf87253-ec46-4eb5-ae6a-431442424c10' {{(pid=156391) create_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1040}}
Jul 02 21:21:52.623911 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:52.789600 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_replicated_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:915}}
Jul 02 21:21:52.794147 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] extra_specs: {'volume_backend_name': 'ceph'} {{(pid=156391) _is_multiattach_type /opt/stack/cinder/cinder/volume/drivers/rbd.py:924}}
Jul 02 21:21:52.802296 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (dc9a72df-7a24-4c44-a224-0a0dfc36c9b3) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'volume_name': 'volume-1cf87253-ec46-4eb5-ae6a-431442424c10', 'volume_size': 2}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:52.821635 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bb9780af-0387-4954-a5ba-41926ed3fbd9) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:52.858707 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Volume volume-1cf87253-ec46-4eb5-ae6a-431442424c10 (1cf87253-ec46-4eb5-ae6a-431442424c10): created successfully
Jul 02 21:21:52.863735 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeOnFinishTask;volume:create, create.end' (bb9780af-0387-4954-a5ba-41926ed3fbd9) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:52.874386 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Flow 'volume_create_manager' (bf7c7c88-c6eb-4826-afac-549081bc9995) transitioned into state 'SUCCESS' from state 'RUNNING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:52.907077 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-a49c38f5-92dd-4497-aa11-df40a5cdd294 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Created volume successfully.
Jul 02 21:21:53.743175 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-3c0a8a47-4e6e-42c5-85b1-cc377a53249f tempest-VolumesSnapshotNegativeTestJSON-718827360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:54.287287 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-3c0a8a47-4e6e-42c5-85b1-cc377a53249f tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Create snapshot completed successfully
Jul 02 21:21:55.004309 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-fc77b2a0-199c-415e-9adb-4ed0a08baeb5 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Acquiring lock "cinder-d388160e-5906-4f52-9c72-427e03093de5-delete_snapshot" by "delete_snapshot" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:55.016608 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-fc77b2a0-199c-415e-9adb-4ed0a08baeb5 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Lock "cinder-d388160e-5906-4f52-9c72-427e03093de5-delete_snapshot" acquired by "delete_snapshot" :: waited 0.012s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:55.035719 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-fc77b2a0-199c-415e-9adb-4ed0a08baeb5 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:55.430028 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-fc77b2a0-199c-415e-9adb-4ed0a08baeb5 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Created reservations ['d4837c25-c344-41ba-b697-832680dfefb4', '72fa4370-c86e-432d-9700-9a6726d22ffc', 'c1055824-1d96-45b1-b136-67788de15862', '662d512a-1a68-4373-8f55-b35e866851dc'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:55.485085 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-fc77b2a0-199c-415e-9adb-4ed0a08baeb5 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Delete snapshot completed successfully.
Jul 02 21:21:55.491364 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-fc77b2a0-199c-415e-9adb-4ed0a08baeb5 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Lock "cinder-d388160e-5906-4f52-9c72-427e03093de5-delete_snapshot" released by "delete_snapshot" :: held 0.475s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:56.228833 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Acquiring lock "cinder-1cf87253-ec46-4eb5-ae6a-431442424c10-delete_volume" by "delete_volume" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:21:56.250728 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Lock "cinder-1cf87253-ec46-4eb5-ae6a-431442424c10-delete_volume" acquired by "delete_volume" :: waited 0.021s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:21:56.401530 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=59,cinder:UPDATE=33,cinder:DELETE=3,cinder:INSERT=5 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:56.402903 ncdv-master-1 cinder-volume[156391]: DEBUG dbcounter [-] [156391] Writing DB stats cinder:SELECT=58,cinder:INSERT=11,cinder:UPDATE=29,cinder:DELETE=2 {{(pid=156391) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}
Jul 02 21:21:56.405149 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:56.511674 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] volume has no backup snaps {{(pid=156391) _delete_backup_snaps /opt/stack/cinder/cinder/volume/drivers/rbd.py:1173}}
Jul 02 21:21:56.512126 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Volume volume-1cf87253-ec46-4eb5-ae6a-431442424c10 is not a clone. {{(pid=156391) _get_clone_info /opt/stack/cinder/cinder/volume/drivers/rbd.py:1201}}
Jul 02 21:21:56.534846 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] deleting rbd volume volume-1cf87253-ec46-4eb5-ae6a-431442424c10 {{(pid=156391) delete_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:1325}}
Jul 02 21:21:57.023598 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.quota [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Created reservations ['916c1135-f9eb-4ad8-9a28-6ec92b59958b', 'b3c6015b-bf69-4a76-84ee-03aeebba54fc', 'a90e79af-4d5e-4635-9ef0-1cbe6d228073', '153b5448-65f3-49ee-9129-97c805d6e5d0'] {{(pid=156391) reserve /opt/stack/cinder/cinder/quota.py:755}}
Jul 02 21:21:57.107159 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:57.180146 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:21:57.272274 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.manager [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Notifying Schedulers of capabilities ... {{(pid=156391) _publish_service_capabilities /opt/stack/cinder/cinder/manager.py:202}}
Jul 02 21:21:57.288435 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.manager [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Deleted volume successfully.
Jul 02 21:21:57.315663 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-0729165a-d370-4387-8fab-ba4a108f70a7 tempest-VolumesSnapshotNegativeTestJSON-718827360 None] Lock "cinder-1cf87253-ec46-4eb5-ae6a-431442424c10-delete_volume" released by "delete_volume" :: held 1.067s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:21:59.676078 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume reschedule parameters: True retry: {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:21:59.719126 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Flow 'volume_create_manager' (b2e4292a-3de3-48db-b473-44af05605db1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:21:59.733444 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (05f0b1d1-4bd5-4492-a567-bf359537bf18) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:59.770550 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (05f0b1d1-4bd5-4492-a567-bf359537bf18) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:00Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:59.775824 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ea478a6b-11a7-4b17-a822-f3dda3064b47) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:59.800518 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ea478a6b-11a7-4b17-a822-f3dda3064b47) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:59.805647 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20ba5a75-fb10-4f87-8f2e-4262082c9963) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:21:59.993520 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20ba5a75-fb10-4f87-8f2e-4262082c9963) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807ca66dc0>}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:21:59.998379 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (68030bf3-6863-4c5f-9af8-1ad0896278e2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.008600 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (68030bf3-6863-4c5f-9af8-1ad0896278e2) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:22:00.022736 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (558ed92f-369c-4d58-9279-a81be470dec1) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.033131 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume 583eb611-48b6-461d-84eb-d9bc3d6c774c: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807ca66dc0>}
Jul 02 21:22:00.033935 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Cloning 583eb611-48b6-461d-84eb-d9bc3d6c774c from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=156391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:22:00.035550 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:22:00.063954 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.026s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:22:00.204095 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 22, 0, 178237)}. {{(pid=156391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:22:00.204647 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=156391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:22:00.205276 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Found cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=156391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}}
Jul 02 21:22:00.212068 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 0.150s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:22:00.224062 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=156391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:22:00.250556 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 22, 0, 236388)}. {{(pid=156391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:22:00.250985 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=156391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:22:00.251452 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Creating from source image-volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45 {{(pid=156391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}}
Jul 02 21:22:00.295903 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:22:00.440348 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] creating snapshot='volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap' {{(pid=156391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:22:00.498647 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (558ed92f-369c-4d58-9279-a81be470dec1) transitioned into state 'FAILURE' from state 'RUNNING'
Jul 02 21:22:00.498647 ncdv-master-1 cinder-volume[156391]: 5 predecessors (most recent first):
Jul 02 21:22:00.498647 ncdv-master-1 cinder-volume[156391]:   Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f807ca33d30>, 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:00Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)}, 'provides': None}
Jul 02 21:22:00.499007 ncdv-master-1 cinder-volume[156391]:   |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f807ca33d30>, 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:00Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf), 'request_spec': RequestSpec(CG_backend=<?>,availability_zones=['nova'],backup_id=None,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=e482b755-1c1a-4758-a7c0-b3edf1610541,operation=<?>,resource_backend=<?>,snapshot_id=None,source_replicaid=<?>,source_volid=None,volume=Volume(583eb611-48b6-461d-84eb-d9bc3d6c774c),volume_id=583eb611-48b6-461d-84eb-d9bc3d6c774c,volume_properties=VolumeProperties,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf))}, 'provides': {'status': 'creating', 'type': 'image', 'volume_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807ca66dc0>}}
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]:      |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'filter_properties': {'retry': {'num_attempts': 1, 'backends': ['ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph']}, 'request_spec': {'consistencygroup_id': None, 'group_id': None, 'cgsnapshot_id': None, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'snapshot_id': None, 'source_volid': None, 'volume_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume': {'id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', '_name_id': None, 'ec2_id': None, 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'project_id': '9d551fd6c0ca463fb120f6a0bc6d7431', 'use_quota': True, 'snapshot_id': None, 'cluster_name': None, 'host': None, 'size': 1, 'availability_zone': 'nova', 'status': 'creating', 'attach_status': 'detached', 'migration_status': None, 'scheduled_at': None, 'launched_at': None, 'terminated_at': None, 'display_name': 'tempest-VolumesCloneTest-Volume-1588529928', 'display_description': None, 'provider_id': None, 'provider_location': None, 'provider_auth': None, 'provider_geometry': None, 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'source_volid': None, 'encryption_key_id': None, 'consistencygroup_id': None, 'group_id': None, 'deleted': False, 'bootable': False, 'multiattach': False, 'replication_status': None, 'replication_extended_status': None, 'replication_driver_data': None, 'previous_status': None, 'metadata': {}, 'volume_type': {'id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'name': 'ceph', 'description': None, 'is_public': True, 'extra_specs': {'volume_backend_name': 'ceph'}, 'qos_specs_id': None, 'created_at': '2022-07-02T20:50:58.000000', 'updated_at': None, 'deleted_at': None, 'deleted': False}, 'volume_attachment': [], 'service_uuid': None, 'shared_targets': True, 'created_at': '2022-07-02T21:21:59.000000', 'updated_at': None, 'deleted_at': None, 'name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'name_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_metadata': [], 'volume_admin_metadata': [], 'volume_glance_metadata': []}, 'volume_type': {'id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'name': 'ceph', 'description': None, 'is_public': True, 'projects': [], 'extra_specs': {'volume_backend_name': 'ceph'}, 'qos_specs_id': None, 'created_at': '2022-07-02T20:50:58.000000', 'updated_at': None, 'deleted_at': None, 'deleted': False}, 'volume_properties': {'attach_status': 'detached', 'availability_zone': 'nova', 'cgsnapshot_id': None, 'consistencygroup_id': None, 'group_id': None, 'display_description': None, 'display_name': 'tempest-VolumesCloneTest-Volume-1588529928', 'encryption_key_id': None, 'metadata': {}, 'multiattach': False, 'project_id': '9d551fd6c0ca463fb120f6a0bc6d7431', 'qos_specs': None, 'reservations': ['bec16116-c14e-447e-9b9b-e2f28ab90e77', '89bcff90-8383-4aa3-b806-6b199835dfc9', '13e150d6-f424-47bf-854b-b78b344b0dfe', 'ffe9b699-2620-40af-a0ed-a8ceb608b2b9'], 'size': 1, 'snapshot_id': None, 'source_volid': None, 'status': 'creating', 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf'}, 'backup_id': None, 'availability_zones': ['nova'], 'resource_properties': {'attach_status': 'detached', 'availability_zone': 'nova', 'cgsnapshot_id': None, 'consistencygroup_id': None, 'group_id': None, 'display_description': None, 'display_name': 'tempest-VolumesCloneTest-Volume-1588529928', 'encryption_key_id': None, 'metadata': {}, 'multiattach': False, 'project_id': '9d551fd6c0ca463fb120f6a0bc6d7431', 'qos_specs': None, 'reservations': ['bec16116-c14e-447e-9b9b-e2f28ab90e77', '89bcff90-8383-4aa3-b806-6b199835dfc9', '13e150d6-f424-47bf-854b-b78b344b0dfe', 'ffe9b699-2620-40af-a0ed-a8ceb608b2b9'], 'size': 1, 'snapshot_id': None, 'source_volid': None, 'status': 'creating', 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf'}}, 'config_options': {}, 'volume_type': VolumeType(created_at=2022-07-02T20:50:58Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='ceph'},id=54be58df-6f85-4cf8-9fef-8fb5554c82cf,is_public=True,name='ceph',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), 'resource_type': VolumeType(created_at=2022-07-02T20:50:58Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='ceph'},id=54be58df-6f85-4cf8-9fef-8fb5554c82cf,is_public=True,name='ceph',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), 'size': 1, 'availability_zone': 'nova', 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'metadata': {}, 'qos_specs': None}, 'request_spec': RequestSpec(CG_backend=<?>,availability_zones=['nova'],backup_id=None,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=e482b755-1c1a-4758-a7c0-b3edf1610541,operation=<?>,resource_backend=<?>,snapshot_id=None,source_replicaid=<?>,source_volid=None,volume=Volume(583eb611-48b6-461d-84eb-d9bc3d6c774c),volume_id=583eb611-48b6-461d-84eb-d9bc3d6c774c,volume_properties=VolumeProperties,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf)), 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:00Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf), 'context': <cinder.context.RequestContext object at 0x7f807ca33d30>}, 'provides': None}
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]:         |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f807ca33d30>, 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:00Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:00Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:00Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)}
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]:            |__Flow 'volume_create_manager': rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager Traceback (most recent call last):
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 782, in create_cloned_volume
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     src_volume.create_snap(clone_snap)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 3514, in rbd.Image.create_snap
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager rbd.ReadOnlyImage: [errno 30] RBD read-only image (error creating snapshot b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager 
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager During handling of the above exception, another exception occurred:
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager 
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager Traceback (most recent call last):
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/executor.py", line 52, in _execute_task
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     result = task.execute(**arguments)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1235, in execute
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self._create_from_image(context,
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/utils.py", line 669, in _wrapper
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     return r(f, *args, **kwargs)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 423, in __call__
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     do = self.iter(retry_state=retry_state)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 360, in iter
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     return fut.result()
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     return self.__get_result()
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     raise self._exception
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 426, in __call__
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     result = fn(*args, **kwargs)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1123, in _create_from_image
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self._create_from_image_cache_or_download(
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 942, in _create_from_image_cache_or_download
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update, cloned = self._create_from_image_cache(
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 841, in _create_from_image_cache
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self._create_from_source_volume(
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 657, in _create_from_source_volume
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self.driver.create_cloned_volume(volume, srcvol_ref)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 793, in create_cloned_volume
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     src_volume.unprotect_snap(clone_snap)
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 3641, in rbd.Image.unprotect_snap
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:00.499236 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager 
Jul 02 21:22:00.560941 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (558ed92f-369c-4d58-9279-a81be470dec1) transitioned into state 'REVERTING' from state 'FAILURE' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.571757 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (558ed92f-369c-4d58-9279-a81be470dec1) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:00.583356 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (68030bf3-6863-4c5f-9af8-1ad0896278e2) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.591777 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (68030bf3-6863-4c5f-9af8-1ad0896278e2) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:00.602134 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20ba5a75-fb10-4f87-8f2e-4262082c9963) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.622007 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (20ba5a75-fb10-4f87-8f2e-4262082c9963) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:00.648454 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ea478a6b-11a7-4b17-a822-f3dda3064b47) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.653883 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Updating volume 583eb611-48b6-461d-84eb-d9bc3d6c774c with {'scheduled_at': datetime.datetime(2022, 7, 2, 21, 22, 0, 653364), 'host': None}. {{(pid=156391) _pre_reschedule /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:138}}
Jul 02 21:22:00.697431 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume 583eb611-48b6-461d-84eb-d9bc3d6c774c: re-scheduling SchedulerAPI.create_volume attempt 1 due to [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap') {{(pid=156391) _reschedule /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:161}}
Jul 02 21:22:00.745647 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume 583eb611-48b6-461d-84eb-d9bc3d6c774c: re-scheduled {{(pid=156391) _post_reschedule /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:178}}
Jul 02 21:22:00.746319 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:22:00.833123 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] volume volume-583eb611-48b6-461d-84eb-d9bc3d6c774c no longer exists in backend
Jul 02 21:22:00.851649 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (ea478a6b-11a7-4b17-a822-f3dda3064b47) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
Jul 02 21:22:00.856264 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (05f0b1d1-4bd5-4492-a567-bf359537bf18) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:00.873859 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (05f0b1d1-4bd5-4492-a567-bf359537bf18) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:00.881882 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Flow 'volume_create_manager' (b2e4292a-3de3-48db-b473-44af05605db1) transitioned into state 'REVERTED' from state 'RUNNING'
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Exception during message handling: rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 782, in create_cloned_volume
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     src_volume.create_snap(clone_snap)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3514, in rbd.Image.create_snap
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server rbd.ReadOnlyImage: [errno 30] RBD read-only image (error creating snapshot b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return caller(func, *(extras + args), **kw)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/objects/cleanable.py", line 208, in wrapper
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 819, in create_volume
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     _run_flow()
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 811, in _run_flow
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     flow_engine.run()
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/types/failure.py", line 338, in reraise_if_any
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     failures[0].reraise()
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/types/failure.py", line 350, in reraise
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     raise value
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/executor.py", line 52, in _execute_task
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1235, in execute
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self._create_from_image(context,
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/utils.py", line 669, in _wrapper
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return r(f, *args, **kwargs)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 423, in __call__
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     do = self.iter(retry_state=retry_state)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 360, in iter
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return fut.result()
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return self.__get_result()
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     raise self._exception
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 426, in __call__
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = fn(*args, **kwargs)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1123, in _create_from_image
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self._create_from_image_cache_or_download(
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 942, in _create_from_image_cache_or_download
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update, cloned = self._create_from_image_cache(
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 841, in _create_from_image_cache
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self._create_from_source_volume(
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 657, in _create_from_source_volume
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self.driver.create_cloned_volume(volume, srcvol_ref)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 793, in create_cloned_volume
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     src_volume.unprotect_snap(clone_snap)
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3641, in rbd.Image.unprotect_snap
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:00.903961 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:22:01.145865 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume reschedule parameters: True retry: {'num_attempts': 2, 'backends': ['ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:22:01.239889 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Flow 'volume_create_manager' (00ff22fa-edc1-4d69-9a03-3f2cc9661e11) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}
Jul 02 21:22:01.264793 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1720f2a2-a3db-45a0-8c37-590202ac0289) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.333715 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1720f2a2-a3db-45a0-8c37-590202ac0289) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:01Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:22:01.358378 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (42d2a060-bef5-4a3f-826e-363ffeae416e) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.363128 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (42d2a060-bef5-4a3f-826e-363ffeae416e) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:22:01.368155 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74891a04-05bd-4aeb-a5fb-383d604b68d2) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.543114 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74891a04-05bd-4aeb-a5fb-383d604b68d2) transitioned into state 'SUCCESS' from state 'RUNNING' with result '{'status': 'creating', 'type': 'image', 'volume_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807c9f38b0>}' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:22:01.552587 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3fc0c304-cdaf-4f94-8179-00c2b5439848) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.558217 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3fc0c304-cdaf-4f94-8179-00c2b5439848) transitioned into state 'SUCCESS' from state 'RUNNING' with result 'None' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:178}}
Jul 02 21:22:01.566019 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b7515e2d-68ca-4756-b0bf-a1dde02243da) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.567240 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume 583eb611-48b6-461d-84eb-d9bc3d6c774c: being created as image with specification: {'status': 'creating', 'volume_name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807c9f38b0>}
Jul 02 21:22:01.567775 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Cloning 583eb611-48b6-461d-84eb-d9bc3d6c774c from image e482b755-1c1a-4758-a7c0-b3edf1610541  at location (None, None). {{(pid=156391) _create_from_image /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1069}}
Jul 02 21:22:01.568467 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Acquiring lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" by "_prepare_image_cache_entry" {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:192}}
Jul 02 21:22:01.598687 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" acquired by "_prepare_image_cache_entry" :: waited 0.030s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:195}}
Jul 02 21:22:01.627837 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 22, 1, 619880)}. {{(pid=156391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:22:01.629360 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=156391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:22:01.630113 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Found cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=156391) _prepare_image_cache_entry /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:884}}
Jul 02 21:22:01.645271 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.coordination [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Lock "cinder-e482b755-1c1a-4758-a7c0-b3edf1610541" released by "_prepare_image_cache_entry" :: held 0.047s {{(pid=156391) _synchronized /opt/stack/cinder/cinder/coordination.py:204}}
Jul 02 21:22:01.647554 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Attempting to retrieve cache entry for image = e482b755-1c1a-4758-a7c0-b3edf1610541 on host ncdv-master-1@ceph#ceph. {{(pid=156391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:825}}
Jul 02 21:22:01.656001 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Found image-volume cache entry: {'id': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'volume_id': '12fc5816-9c32-4ebe-a0f8-30a641fe4f45', 'host': 'ncdv-master-1@ceph#ceph', 'size': 1, 'image_updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29), 'last_used': datetime.datetime(2022, 7, 2, 21, 22, 1, 652644)}. {{(pid=156391) get_entry /opt/stack/cinder/cinder/image/cache.py:76}}
Jul 02 21:22:01.656464 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.image.cache [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Image-volume cache entry image_update_at = 2022-07-02 20:51:29+00:00, requested image updated_at = 2022-07-02 20:51:29+00:00. {{(pid=156391) _should_update_entry /opt/stack/cinder/cinder/image/cache.py:259}}
Jul 02 21:22:01.656890 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Creating from source image-volume 12fc5816-9c32-4ebe-a0f8-30a641fe4f45 {{(pid=156391) _create_from_image_cache /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:839}}
Jul 02 21:22:01.677680 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:22:01.782100 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] creating snapshot='volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap' {{(pid=156391) create_cloned_volume /opt/stack/cinder/cinder/volume/drivers/rbd.py:779}}
Jul 02 21:22:01.808846 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b7515e2d-68ca-4756-b0bf-a1dde02243da) transitioned into state 'FAILURE' from state 'RUNNING'
Jul 02 21:22:01.808846 ncdv-master-1 cinder-volume[156391]: 5 predecessors (most recent first):
Jul 02 21:22:01.808846 ncdv-master-1 cinder-volume[156391]:   Atom 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f807c9d6160>, 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:01Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)}, 'provides': None}
Jul 02 21:22:01.808846 ncdv-master-1 cinder-volume[156391]:   |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f807c9d6160>, 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:01Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf), 'request_spec': RequestSpec(CG_backend=<?>,availability_zones=['nova'],backup_id=None,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=e482b755-1c1a-4758-a7c0-b3edf1610541,operation=<?>,resource_backend=<?>,snapshot_id=None,source_replicaid=<?>,source_volid=None,volume=Volume(583eb611-48b6-461d-84eb-d9bc3d6c774c),volume_id=583eb611-48b6-461d-84eb-d9bc3d6c774c,volume_properties=VolumeProperties,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf))}, 'provides': {'status': 'creating', 'type': 'image', 'volume_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_size': 1, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'image_location': (None, None), 'image_meta': {'name': 'ubuntu-16', 'disk_format': 'vhdx', 'container_format': 'bare', 'visibility': 'public', 'size': 645922816, 'virtual_size': 734003200, 'status': 'active', 'checksum': '318f990048c8a194829e52141534610d', 'protected': False, 'min_ram': 0, 'min_disk': 0, 'owner': 'd4c9de22f7474b6f9c2ca908dc6ecd1f', 'os_hidden': False, 'os_hash_algo': 'sha512', 'os_hash_value': '1aaa7a04b14d2fb410b8803fefeb767485c32e2da475b52a8839eec757ebef67d8611e4f699200324e39744ee6949dd3c2a4f6ecccb0c2480706ec5cb0fc5a53', 'id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'created_at': datetime.datetime(2022, 7, 2, 20, 51, 14, tzinfo=datetime.timezone.utc), 'updated_at': datetime.datetime(2022, 7, 2, 20, 51, 29, tzinfo=datetime.timezone.utc), 'tags': [], 'file': '/v2/images/e482b755-1c1a-4758-a7c0-b3edf1610541/file', 'properties': {'hw_rng_model': 'virtio', 'owner_specified.openstack.md5': '', 'owner_specified.openstack.object': 'images/ubuntu-16', 'owner_specified.openstack.sha256': ''}}, 'image_service': <cinder.image.glance.GlanceImageService object at 0x7f807c9f38b0>}}
Jul 02 21:22:01.808846 ncdv-master-1 cinder-volume[156391]:      |__Atom 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'filter_properties': {'request_spec': {'consistencygroup_id': None, 'group_id': None, 'cgsnapshot_id': None, 'image_id': 'e482b755-1c1a-4758-a7c0-b3edf1610541', 'snapshot_id': None, 'source_volid': None, 'volume_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume': {'id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', '_name_id': None, 'ec2_id': None, 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'project_id': '9d551fd6c0ca463fb120f6a0bc6d7431', 'use_quota': True, 'snapshot_id': None, 'cluster_name': None, 'host': None, 'size': 1, 'availability_zone': 'nova', 'status': 'creating', 'attach_status': 'detached', 'migration_status': None, 'scheduled_at': None, 'launched_at': None, 'terminated_at': None, 'display_name': 'tempest-VolumesCloneTest-Volume-1588529928', 'display_description': None, 'provider_id': None, 'provider_location': None, 'provider_auth': None, 'provider_geometry': None, 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'source_volid': None, 'encryption_key_id': None, 'consistencygroup_id': None, 'group_id': None, 'deleted': False, 'bootable': False, 'multiattach': False, 'replication_status': None, 'replication_extended_status': None, 'replication_driver_data': None, 'previous_status': None, 'metadata': {}, 'admin_metadata': {}, 'glance_metadata': {}, 'volume_type': {'id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'name': 'ceph', 'description': None, 'is_public': True, 'extra_specs': {'volume_backend_name': 'ceph'}, 'qos_specs_id': None, 'created_at': '2022-07-02T20:50:58.000000', 'updated_at': None, 'deleted_at': None, 'deleted': False}, 'volume_attachment': [], 'service_uuid': None, 'shared_targets': True, 'created_at': '2022-07-02T21:21:59.000000', 'updated_at': None, 'deleted_at': None, 'name': 'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c', 'name_id': '583eb611-48b6-461d-84eb-d9bc3d6c774c', 'volume_metadata': [], 'volume_admin_metadata': [], 'volume_glance_metadata': []}, 'volume_type': {'id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf', 'name': 'ceph', 'description': None, 'is_public': True, 'projects': [], 'extra_specs': {'volume_backend_name': 'ceph'}, 'qos_specs_id': None, 'created_at': '2022-07-02T20:50:58.000000', 'updated_at': None, 'deleted_at': None, 'deleted': False}, 'volume_properties': {'attach_status': 'detached', 'availability_zone': 'nova', 'cgsnapshot_id': None, 'consistencygroup_id': None, 'group_id': None, 'display_description': None, 'display_name': 'tempest-VolumesCloneTest-Volume-1588529928', 'encryption_key_id': None, 'metadata': {}, 'multiattach': False, 'project_id': '9d551fd6c0ca463fb120f6a0bc6d7431', 'qos_specs': None, 'reservations': ['bec16116-c14e-447e-9b9b-e2f28ab90e77', '89bcff90-8383-4aa3-b806-6b199835dfc9', '13e150d6-f424-47bf-854b-b78b344b0dfe', 'ffe9b699-2620-40af-a0ed-a8ceb608b2b9'], 'size': 1, 'snapshot_id': None, 'source_volid': None, 'status': 'creating', 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf'}, 'backup_id': None, 'availability_zones': ['nova'], 'resource_properties': {'attach_status': 'detached', 'availability_zone': 'nova', 'cgsnapshot_id': None, 'consistencygroup_id': None, 'group_id': None, 'display_description': None, 'display_name': 'tempest-VolumesCloneTest-Volume-1588529928', 'encryption_key_id': None, 'metadata': {}, 'multiattach': False, 'project_id': '9d551fd6c0ca463fb120f6a0bc6d7431', 'qos_specs': None, 'reservations': ['bec16116-c14e-447e-9b9b-e2f28ab90e77', '89bcff90-8383-4aa3-b806-6b199835dfc9', '13e150d6-f424-47bf-854b-b78b344b0dfe', 'ffe9b699-2620-40af-a0ed-a8ceb608b2b9'], 'size': 1, 'snapshot_id': None, 'source_volid': None, 'status': 'creating', 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'volume_type_id': '54be58df-6f85-4cf8-9fef-8fb5554c82cf'}}, 'config_options': {}, 'volume_type': VolumeType(created_at=2022-07-02T20:50:58Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='ceph'},id=54be58df-6f85-4cf8-9fef-8fb5554c82cf,is_public=True,name='ceph',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), 'resource_type': VolumeType(created_at=2022-07-02T20:50:58Z,deleted=False,deleted_at=None,description=None,extra_specs={volume_backend_name='ceph'},id=54be58df-6f85-4cf8-9fef-8fb5554c82cf,is_public=True,name='ceph',projects=[],qos_specs=<?>,qos_specs_id=None,updated_at=None), 'size': 1, 'availability_zone': 'nova', 'user_id': '5a6e0a0f3a6e4856b0ab47fc81106454', 'metadata': {}, 'qos_specs': None, 'retry': {'num_attempts': 2, 'backends': ['ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph']}}, 'request_spec': RequestSpec(CG_backend=<?>,availability_zones=['nova'],backup_id=None,cgsnapshot_id=None,consistencygroup_id=None,group_backend=<?>,group_id=None,image_id=e482b755-1c1a-4758-a7c0-b3edf1610541,operation=<?>,resource_backend=<?>,snapshot_id=None,source_replicaid=<?>,source_volid=None,volume=Volume(583eb611-48b6-461d-84eb-d9bc3d6c774c),volume_id=583eb611-48b6-461d-84eb-d9bc3d6c774c,volume_properties=VolumeProperties,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf)), 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:01Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf), 'context': <cinder.context.RequestContext object at 0x7f807c9d6160>}, 'provides': None}
Jul 02 21:22:01.808846 ncdv-master-1 cinder-volume[156391]:         |__Atom 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' {'intention': 'EXECUTE', 'state': 'SUCCESS', 'requires': {'context': <cinder.context.RequestContext object at 0x7f807c9d6160>, 'volume': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:01Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)}, 'provides': Volume(_name_id=None,admin_metadata=<?>,attach_status='detached',availability_zone='nova',bootable=False,cluster=<?>,cluster_name=None,consistencygroup=<?>,consistencygroup_id=None,created_at=2022-07-02T21:21:59Z,deleted=False,deleted_at=None,display_description=None,display_name='tempest-VolumesCloneTest-Volume-1588529928',ec2_id=None,encryption_key_id=None,glance_metadata=<?>,group=<?>,group_id=None,host='ncdv-master-1@ceph#ceph',id=583eb611-48b6-461d-84eb-d9bc3d6c774c,launched_at=None,metadata={},migration_status=None,multiattach=False,previous_status=None,project_id='9d551fd6c0ca463fb120f6a0bc6d7431',provider_auth=None,provider_geometry=None,provider_id=None,provider_location=None,replication_driver_data=None,replication_extended_status=None,replication_status=None,scheduled_at=2022-07-02T21:22:01Z,service_uuid=None,shared_targets=True,size=1,snapshot_id=None,snapshots=<?>,source_volid=None,status='creating',terminated_at=None,updated_at=2022-07-02T21:22:01Z,use_quota=True,user_id='5a6e0a0f3a6e4856b0ab47fc81106454',volume_attachment=VolumeAttachmentList,volume_type=VolumeType(54be58df-6f85-4cf8-9fef-8fb5554c82cf),volume_type_id=54be58df-6f85-4cf8-9fef-8fb5554c82cf)}
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]:            |__Flow 'volume_create_manager': rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager Traceback (most recent call last):
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 782, in create_cloned_volume
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     src_volume.create_snap(clone_snap)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 3514, in rbd.Image.create_snap
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager rbd.ReadOnlyImage: [errno 30] RBD read-only image (error creating snapshot b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager 
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager During handling of the above exception, another exception occurred:
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager 
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager Traceback (most recent call last):
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/executor.py", line 52, in _execute_task
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     result = task.execute(**arguments)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1235, in execute
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self._create_from_image(context,
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/utils.py", line 669, in _wrapper
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     return r(f, *args, **kwargs)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 423, in __call__
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     do = self.iter(retry_state=retry_state)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 360, in iter
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     return fut.result()
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     return self.__get_result()
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     raise self._exception
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 426, in __call__
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     result = fn(*args, **kwargs)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1123, in _create_from_image
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self._create_from_image_cache_or_download(
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 942, in _create_from_image_cache_or_download
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update, cloned = self._create_from_image_cache(
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 841, in _create_from_image_cache
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self._create_from_source_volume(
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 657, in _create_from_source_volume
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     model_update = self.driver.create_cloned_volume(volume, srcvol_ref)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 793, in create_cloned_volume
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager     src_volume.unprotect_snap(clone_snap)
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager   File "rbd.pyx", line 3641, in rbd.Image.unprotect_snap
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:01.809661 ncdv-master-1 cinder-volume[156391]: ERROR cinder.volume.manager 
Jul 02 21:22:01.857480 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b7515e2d-68ca-4756-b0bf-a1dde02243da) transitioned into state 'REVERTING' from state 'FAILURE' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.881780 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.CreateVolumeFromSpecTask;volume:create' (b7515e2d-68ca-4756-b0bf-a1dde02243da) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:01.893082 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3fc0c304-cdaf-4f94-8179-00c2b5439848) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.907522 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.NotifyVolumeActionTask;volume:create, create.start' (3fc0c304-cdaf-4f94-8179-00c2b5439848) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:01.925250 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74891a04-05bd-4aeb-a5fb-383d604b68d2) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.929353 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeSpecTask;volume:create' (74891a04-05bd-4aeb-a5fb-383d604b68d2) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:01.947807 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (42d2a060-bef5-4a3f-826e-363ffeae416e) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:01.950160 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Updating volume 583eb611-48b6-461d-84eb-d9bc3d6c774c with {'scheduled_at': datetime.datetime(2022, 7, 2, 21, 22, 1, 949499), 'host': None}. {{(pid=156391) _pre_reschedule /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:138}}
Jul 02 21:22:01.973961 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume 583eb611-48b6-461d-84eb-d9bc3d6c774c: re-scheduling SchedulerAPI.create_volume attempt 2 due to [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap') {{(pid=156391) _reschedule /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:161}}
Jul 02 21:22:02.017128 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume 583eb611-48b6-461d-84eb-d9bc3d6c774c: re-scheduled {{(pid=156391) _post_reschedule /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:178}}
Jul 02 21:22:02.017811 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] connecting to cinder@ceph (conf=/etc/ceph/ceph.conf, timeout=-1). {{(pid=156391) _do_conn /opt/stack/cinder/cinder/volume/drivers/rbd.py:523}}
Jul 02 21:22:02.115791 ncdv-master-1 cinder-volume[156391]: INFO cinder.volume.drivers.rbd [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] volume volume-583eb611-48b6-461d-84eb-d9bc3d6c774c no longer exists in backend
Jul 02 21:22:02.126329 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.OnFailureRescheduleTask;volume:create' (42d2a060-bef5-4a3f-826e-363ffeae416e) transitioned into state 'REVERTED' from state 'REVERTING' with result 'True'
Jul 02 21:22:02.145004 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1720f2a2-a3db-45a0-8c37-590202ac0289) transitioned into state 'REVERTING' from state 'SUCCESS' {{(pid=156391) _task_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:190}}
Jul 02 21:22:02.152899 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Task 'cinder.volume.flows.manager.create_volume.ExtractVolumeRefTask;volume:create' (1720f2a2-a3db-45a0-8c37-590202ac0289) transitioned into state 'REVERTED' from state 'REVERTING' with result 'None'
Jul 02 21:22:02.171314 ncdv-master-1 cinder-volume[156391]: WARNING cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Flow 'volume_create_manager' (00ff22fa-edc1-4d69-9a03-3f2cc9661e11) transitioned into state 'REVERTED' from state 'RUNNING'
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Exception during message handling: rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 782, in create_cloned_volume
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     src_volume.create_snap(clone_snap)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3514, in rbd.Image.create_snap
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server rbd.ReadOnlyImage: [errno 30] RBD read-only image (error creating snapshot b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap' from b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45')
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server During handling of the above exception, another exception occurred:
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server Traceback (most recent call last):
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/server.py", line 165, in _process_incoming
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     res = self.dispatcher.dispatch(message)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 309, in dispatch
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return self._do_dispatch(endpoint, method, ctxt, args)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/oslo_messaging/rpc/dispatcher.py", line 229, in _do_dispatch
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = func(ctxt, **new_args)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return caller(func, *(extras + args), **kw)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/objects/cleanable.py", line 208, in wrapper
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = f(*args, **kwargs)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 819, in create_volume
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     _run_flow()
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/manager.py", line 811, in _run_flow
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     flow_engine.run()
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/engine.py", line 247, in run
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     for _state in self.run_iter(timeout=timeout):
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/engine.py", line 340, in run_iter
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     failure.Failure.reraise_if_any(er_failures)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/types/failure.py", line 338, in reraise_if_any
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     failures[0].reraise()
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/types/failure.py", line 350, in reraise
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     raise value
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/taskflow/engines/action_engine/executor.py", line 52, in _execute_task
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = task.execute(**arguments)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1235, in execute
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self._create_from_image(context,
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/utils.py", line 669, in _wrapper
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return r(f, *args, **kwargs)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 423, in __call__
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     do = self.iter(retry_state=retry_state)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 360, in iter
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return fut.result()
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 437, in result
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     return self.__get_result()
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/lib/python3.8/concurrent/futures/_base.py", line 389, in __get_result
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     raise self._exception
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/usr/local/lib/python3.8/dist-packages/tenacity/__init__.py", line 426, in __call__
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     result = fn(*args, **kwargs)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 1123, in _create_from_image
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self._create_from_image_cache_or_download(
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 942, in _create_from_image_cache_or_download
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update, cloned = self._create_from_image_cache(
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 841, in _create_from_image_cache
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self._create_from_source_volume(
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/flows/manager/create_volume.py", line 657, in _create_from_source_volume
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     model_update = self.driver.create_cloned_volume(volume, srcvol_ref)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "/opt/stack/cinder/cinder/volume/drivers/rbd.py", line 793, in create_cloned_volume
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server     src_volume.unprotect_snap(clone_snap)
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 2770, in rbd.requires_not_closed.wrapper
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server   File "rbd.pyx", line 3641, in rbd.Image.unprotect_snap
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server rbd.ImageNotFound: [errno 2] RBD image not found (error unprotecting snapshot b'volume-12fc5816-9c32-4ebe-a0f8-30a641fe4f45'@b'volume-583eb611-48b6-461d-84eb-d9bc3d6c774c.clone_snap')
Jul 02 21:22:02.173494 ncdv-master-1 cinder-volume[156391]: ERROR oslo_messaging.rpc.server 
Jul 02 21:22:02.309274 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.flows.manager.create_volume [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Volume reschedule parameters: True retry: {'num_attempts': 3, 'backends': ['ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph'], 'hosts': ['ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph', 'ncdv-master-1@ceph#ceph']} {{(pid=156391) get_flow /opt/stack/cinder/cinder/volume/flows/manager/create_volume.py:1366}}
Jul 02 21:22:02.350620 ncdv-master-1 cinder-volume[156391]: DEBUG cinder.volume.manager [None req-f1b3c623-30e0-4f16-a8da-68457f66b125 tempest-VolumesCloneTest-1244872301 None] Flow 'volume_create_manager' (0b620645-8fbf-4ee1-aa21-aa23974f91ab) transitioned into state 'RUNNING' from state 'PENDING' {{(pid=156391) _flow_receiver /usr/local/lib/python3.8/dist-packages/taskflow/listeners/logging.py:141}}